Introduction

In this Notebook, I have tried to build a model which predicts the age of a person based on his or her face.

The techniques I have used to build this Image Regression model are based on Fastai's CNN models.

The dataset I have used for this task are as follows:

UTK Face Data
Appa Real Face Data
IMDB Wiki Face Data

In the course of this analysis, I have learnt many things:

How to augment the data using Fastai's Image augmentation techinues.
Loading and using Cadene's Pretrained Models (https://github.com/Cadene/pretrained-models.pytorch)
How to use Image resizing technique which basically refers to gradually increase the size of image while training. This helps in achieving greater prediction accuracy.
Discriminative Layers Training technique

Import Libraries

# This Python 3 environment comes with many helpful analytics libraries installed
# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python
# For example, here's several helpful packages to load in 

import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)

# Input data files are available in the "../input/" directory.
# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory

import os
print(os.listdir("../input"))

# Any results you write to the current directory are saved as output.

import warnings
warnings.filterwarnings('ignore')
import gc
gc.collect()

['picture-2', 'picture4', 'wiki-face-data', 'picture-1', 'utk-face-cropped', 'picture-6', 'imdb-wiki-faces-dataset', 'picture-5', 'picture-3', 'appa-real-face-cropped']

0

!pip install pretrainedmodels

%reload_ext autoreload
%autoreload 2
%matplotlib inline

!pip install fastai==1.0.57
import fastai

from fastai import *
from fastai.vision import *

from torchvision.models import *
import pretrainedmodels

from utils import *
import sys

from fastai.callbacks.hooks import *

from fastai.callbacks.tracker import EarlyStoppingCallback
from fastai.callbacks.tracker import SaveModelCallback

path_wiki = Path('../input/wiki-face-data/wiki_crop/wiki_crop/')
path_imdb = Path('../input/imdb-wiki-faces-dataset/imdb_crop/imdb_crop/')

Collecting pretrainedmodels
  Downloading https://files.pythonhosted.org/packages/84/0e/be6a0e58447ac16c938799d49bfb5fb7a80ac35e137547fc6cee2c08c4cf/pretrainedmodels-0.7.4.tar.gz (58kB)
     |████████████████████████████████| 61kB 4.9MB/s eta 0:00:011
Requirement already satisfied: torch in /opt/conda/lib/python3.6/site-packages (from pretrainedmodels) (1.2.0)
Requirement already satisfied: torchvision in /opt/conda/lib/python3.6/site-packages (from pretrainedmodels) (0.4.0a0+6b959ee)
Requirement already satisfied: munch in /opt/conda/lib/python3.6/site-packages (from pretrainedmodels) (2.3.2)
Requirement already satisfied: tqdm in /opt/conda/lib/python3.6/site-packages (from pretrainedmodels) (4.36.1)
Requirement already satisfied: numpy in /opt/conda/lib/python3.6/site-packages (from torch->pretrainedmodels) (1.16.4)
Requirement already satisfied: six in /opt/conda/lib/python3.6/site-packages (from torchvision->pretrainedmodels) (1.12.0)
Requirement already satisfied: pillow>=4.1.1 in /opt/conda/lib/python3.6/site-packages (from torchvision->pretrainedmodels) (5.4.1)
Building wheels for collected packages: pretrainedmodels
  Building wheel for pretrainedmodels (setup.py) ... done
  Created wheel for pretrainedmodels: filename=pretrainedmodels-0.7.4-cp36-none-any.whl size=60963 sha256=b82608a0218b0ec36665a9794d3c75dedcac7ece90c42fab8c881ef88f2d2389
  Stored in directory: /tmp/.cache/pip/wheels/69/df/63/62583c096289713f22db605aa2334de5b591d59861a02c2ecd
Successfully built pretrainedmodels
Installing collected packages: pretrainedmodels
Successfully installed pretrainedmodels-0.7.4
Requirement already satisfied: fastai==1.0.57 in /opt/conda/lib/python3.6/site-packages (1.0.57)
Requirement already satisfied: packaging in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (19.2)
Requirement already satisfied: scipy in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (1.2.1)
Requirement already satisfied: beautifulsoup4 in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (4.8.0)
Requirement already satisfied: pyyaml in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (5.1.2)
Requirement already satisfied: spacy>=2.0.18 in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (2.1.8)
Requirement already satisfied: numexpr in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (2.6.9)
Requirement already satisfied: bottleneck in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (1.2.1)
Requirement already satisfied: matplotlib in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (3.0.3)
Requirement already satisfied: typing; python_version < "3.7" in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (3.6.6)
Requirement already satisfied: nvidia-ml-py3 in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (7.352.0)
Requirement already satisfied: Pillow in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (5.4.1)
Requirement already satisfied: pandas in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (0.25.1)
Requirement already satisfied: torchvision in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (0.4.0a0+6b959ee)
Requirement already satisfied: fastprogress>=0.1.19 in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (0.1.21)
Requirement already satisfied: dataclasses; python_version < "3.7" in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (0.6)
Requirement already satisfied: torch>=1.0.0 in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (1.2.0)
Requirement already satisfied: numpy>=1.15 in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (1.16.4)
Requirement already satisfied: requests in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (2.22.0)
Requirement already satisfied: six in /opt/conda/lib/python3.6/site-packages (from packaging->fastai==1.0.57) (1.12.0)
Requirement already satisfied: pyparsing>=2.0.2 in /opt/conda/lib/python3.6/site-packages (from packaging->fastai==1.0.57) (2.4.2)
Requirement already satisfied: soupsieve>=1.2 in /opt/conda/lib/python3.6/site-packages (from beautifulsoup4->fastai==1.0.57) (1.9.3)
Requirement already satisfied: blis<0.3.0,>=0.2.2 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (0.2.4)
Requirement already satisfied: thinc<7.1.0,>=7.0.8 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (7.0.8)
Requirement already satisfied: plac<1.0.0,>=0.9.6 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (0.9.6)
Requirement already satisfied: srsly<1.1.0,>=0.0.6 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (0.1.0)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (1.0.2)
Requirement already satisfied: wasabi<1.1.0,>=0.2.0 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (0.2.2)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (2.0.2)
Requirement already satisfied: preshed<2.1.0,>=2.0.1 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (2.0.1)
Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.6/site-packages (from matplotlib->fastai==1.0.57) (0.10.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/lib/python3.6/site-packages (from matplotlib->fastai==1.0.57) (1.1.0)
Requirement already satisfied: python-dateutil>=2.1 in /opt/conda/lib/python3.6/site-packages (from matplotlib->fastai==1.0.57) (2.8.0)
Requirement already satisfied: pytz>=2017.2 in /opt/conda/lib/python3.6/site-packages (from pandas->fastai==1.0.57) (2019.2)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.6/site-packages (from requests->fastai==1.0.57) (2019.9.11)
Requirement already satisfied: idna<2.9,>=2.5 in /opt/conda/lib/python3.6/site-packages (from requests->fastai==1.0.57) (2.8)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.6/site-packages (from requests->fastai==1.0.57) (1.24.2)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /opt/conda/lib/python3.6/site-packages (from requests->fastai==1.0.57) (3.0.4)
Requirement already satisfied: tqdm<5.0.0,>=4.10.0 in /opt/conda/lib/python3.6/site-packages (from thinc<7.1.0,>=7.0.8->spacy>=2.0.18->fastai==1.0.57) (4.36.1)
Requirement already satisfied: setuptools in /opt/conda/lib/python3.6/site-packages (from kiwisolver>=1.0.1->matplotlib->fastai==1.0.57) (41.2.0)

import scipy.io
import numpy as np
import pandas as pd
from datetime import datetime, timedelta

Loading the Data

mat_wiki = scipy.io.loadmat('../input/wiki-face-data/wiki_crop/wiki_crop/wiki.mat')
mat_imdb = scipy.io.loadmat('../input/imdb-wiki-faces-dataset/imdb_crop/imdb_crop/imdb.mat')

columns = ["dob", "photo_taken", "full_path", "gender", "name", "face_location", 
           "face_score", "second_face_score", 'celeb_names', 'celeb_id']

instances_wiki = mat_wiki['wiki'][0][0][0].shape[1]
instances_imdb = mat_imdb['imdb'][0][0][0].shape[1]


df_wiki = pd.DataFrame(index = range(0,instances_wiki), columns = columns)
df_imdb = pd.DataFrame(index = range(0,instances_imdb), columns = columns)

for i in mat_wiki:
    if i == "wiki":
        current_array = mat_wiki[i][0][0]
        for j in range(len(current_array)):
            #print(columns[j],": ",current_array[j])
            df_wiki[columns[j]] = pd.DataFrame(current_array[j][0])
            

for i in mat_imdb:
    if i == "imdb":
        current_array = mat_imdb[i][0][0]
        for j in range(len(current_array)):
            #print(columns[j],": ",current_array[j])
            df_imdb[columns[j]] = pd.DataFrame(current_array[j][0])

df_wiki.head()

df_imdb.head()

df_wiki.shape, df_imdb.shape

((62328, 10), (460723, 10))

def datenum_to_datetime(datenum):
    
    try:
        days = datenum % 1
        hours = days % 1 * 24
        minutes = hours % 1 * 60
        seconds = minutes % 1 * 60
        exact_date = datetime.fromordinal(int(datenum)) \
           + timedelta(days=int(days)) \
           + timedelta(hours=int(hours)) \
           + timedelta(minutes=int(minutes)) \
           + timedelta(seconds=round(seconds)) \
           - timedelta(days=366)
    
        return exact_date.year
    
    except(ValueError, TypeError, OverflowError):
        
        return np.nan

df_wiki['date_of_birth'] = df_wiki['dob'].apply(datenum_to_datetime) 
df_imdb['date_of_birth'] = df_imdb['dob'].apply(datenum_to_datetime)

df_wiki['date_of_birth'].value_counts()
df_imdb['date_of_birth'].value_counts()

1969.0    16511
1978.0    16062
1971.0    15344
1976.0    14232
1970.0    14184
          ...  
1879.0        1
2012.0        1
1883.0        1
1888.0        1
1732.0        1
Name: date_of_birth, Length: 128, dtype: int64

df_wiki['age'] = df_wiki['photo_taken'] - df_wiki['date_of_birth']

#remove pictures does not include face
df_wiki = df_wiki[df_wiki['face_score'] != -np.inf]

#some pictures include more than one face, remove them
df_wiki = df_wiki[df_wiki['second_face_score'].isna()]

#check threshold
df_wiki = df_wiki[df_wiki['face_score'] >= 3.5]

df_wiki = df_wiki.drop(columns = ['name','face_score','second_face_score','date_of_birth','face_location'])

#some guys seem to be greater than 100. some of these are paintings. remove these old guys
df_wiki = df_wiki[df_wiki['age'] <= 100]

#some guys seem to be unborn in the data set
df_wiki = df_wiki[df_wiki['age'] > 0]

df_imdb['age'] = df_imdb['photo_taken'] - df_imdb['date_of_birth']

#remove pictures does not include face
df_imdb = df_imdb[df_imdb['face_score'] != -np.inf]

#some pictures include more than one face, remove them
df_imdb = df_imdb[df_imdb['second_face_score'].isna()]

#check threshold
df_imdb = df_imdb[df_imdb['face_score'] >= 3.5]

df_imdb = df_imdb.drop(columns = ['name','face_score','second_face_score','date_of_birth','face_location'])

#some guys seem to be greater than 100. some of these are paintings. remove these old guys
df_imdb = df_imdb[df_imdb['age'] <= 100]

#some guys seem to be unborn in the data set
df_imdb = df_imdb[df_imdb['age'] > 0]

df_wiki.head()

df_wiki.shape, df_imdb.shape

((17096, 7), (73683, 7))

df_wiki['age'] = df_wiki['age'].apply(lambda x: int(x))
df_imdb['age'] = df_imdb['age'].apply(lambda x: int(x))

print(type(df_wiki['age']))
df_wiki['age'].value_counts()
df_imdb['age'].value_counts()

<class 'pandas.core.series.Series'>

31    2757
32    2677
29    2578
33    2545
30    2532
      ... 
96       1
95       1
93       1
92       1
3        1
Name: age, Length: 98, dtype: int64

df_wiki = df_wiki.drop(columns=['dob', 'photo_taken'])
df_imdb = df_imdb.drop(columns=['dob', 'photo_taken'])

df_age_wiki = df_wiki.drop(columns=['gender', 'celeb_names', 'celeb_id'])
df_age_imdb = df_imdb.drop(columns=['gender', 'celeb_names', 'celeb_id'])

df_age_wiki['age'].value_counts()

24    685
25    658
26    658
27    651
23    649
     ... 
97      1
6       1
5       1
94      1
95      1
Name: age, Length: 97, dtype: int64

df_imdb['age'].value_counts()

31    2757
32    2677
29    2578
33    2545
30    2532
      ... 
96       1
95       1
93       1
92       1
3        1
Name: age, Length: 98, dtype: int64

df_age_wiki['full_path'] = df_age_wiki['full_path'].str.get(0)
df_age_imdb['full_path'] = df_age_imdb['full_path'].str.get(0)

df_age_wiki.dropna(axis=0, inplace=True)
df_age_imdb.dropna(axis=0, inplace=True)

df_age_wiki.head()

df_age_imdb.head()

df_age_wiki['age'] = df_age_wiki['age'].apply(lambda x: int(x))
df_age_imdb['age'] = df_age_imdb['age'].apply(lambda x: int(x))

df_age_wiki.shape, df_age_imdb.shape

((17096, 2), (73683, 2))

max_age = df_age_wiki["age"].max(); print(max_age)
min_age = df_age_wiki["age"].min(); print(min_age)

100
1

max_age = df_age_imdb["age"].max(); print(max_age)
min_age = df_age_imdb["age"].min(); print(min_age)

99
1

df_age_wiki.hist()

array([[<matplotlib.axes._subplots.AxesSubplot object at 0x7fcf17fc1278>]], dtype=object)

df_age_imdb.hist()

array([[<matplotlib.axes._subplots.AxesSubplot object at 0x7fcf163a30f0>]], dtype=object)

seed = 42

# # python RNG
# import random
# random.seed(seed)

# # pytorch RNGs
# import torch
# torch.manual_seed(seed)
# torch.backends.cudnn.deterministic = True
# if torch.cuda.is_available(): torch.cuda.manual_seed_all(seed)

# numpy RNG
import numpy as np
np.random.seed(seed)

path = Path('../input/')

path_utk = Path('../input/utk-face-cropped/utkcropped/utkcropped/')

def extract_age(filename):
    return float(filename.stem.split('_')[0])

Preparing the Data for Fastai Models

tfms = get_transforms(max_rotate= 10.,max_zoom=1., max_lighting=0.20, do_flip=False,
                      max_warp=0., xtra_tfms=[flip_lr(), brightness(change=(0.3, 0.60), p=0.7), contrast(scale=(0.5, 2), p=0.7),
                                              crop_pad(size=600, padding_mode='border', row_pct=0.,col_pct=0.),
                                              rand_zoom(scale=(1.,1.5)), rand_crop(),
                                              perspective_warp(magnitude=(-0.1,0.1)),
                                              #jitter(magnitude=(-0.05,0.05), p=0.5),
                                              symmetric_warp(magnitude=(-0.1,0.1)) ])

path_imdb = Path('../input/imdb-wiki-faces-dataset/imdb_crop/imdb_crop/')
path_wiki = Path('../input/wiki-face-data/wiki_crop/wiki_crop/')


data_imdb = ImageList.from_df(df_age_imdb, path_imdb, cols=['full_path'], folder ='.').split_by_rand_pct(0.2, seed=42).label_from_df(label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=128).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)
data_wiki_small = ImageList.from_df(df_age_wiki, path_wiki, cols=['full_path'], folder ='.').split_by_rand_pct(0.2, seed=42).label_from_df(label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=128).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)
data_wiki_big = ImageList.from_df(df_age_wiki, path_wiki, cols=['full_path'], folder ='.').split_by_rand_pct(0.2, seed=42).label_from_df(label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=256).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)

data_utk_small = ImageList.from_folder(path_utk).split_by_rand_pct(0.2, seed=42).label_from_func(extract_age, label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=128).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)
data_utk_big = ImageList.from_folder(path_utk).split_by_rand_pct(0.2, seed=42).label_from_func(extract_age, label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=256).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)

df_appa = pd.read_csv('/kaggle/input/appa-real-face-cropped/labels.csv')
df_appa.head()

df_appa.rename(columns = {"file_name":"full_path", "real_age":"age"}, inplace=True)
df_appa['age'] = df_appa['age'].apply(lambda x: int(float(x)))

path = Path('../input/appa-real-face-cropped/final_files/final_files/')
path_csv = '../input/appa-real-face-cropped/'
path_folder = '../input/appa-real-face-cropped/final_files/final_files/'

data_appa_small = ImageList.from_df(df_appa, path, cols=['full_path'], folder ='.').split_by_rand_pct(0.2, seed=42).label_from_df(label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=128).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)
data_appa_big = ImageList.from_df(df_appa, path, cols=['full_path'], folder ='.').split_by_rand_pct(0.2, seed=42).label_from_df(label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=256).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)

A look at the Images

data_imdb.show_batch(4, figsize=(12,12))

data_wiki_small.show_batch(4, figsize=(12,12))

data_wiki_big.show_batch(4, figsize=(12,12))

data_utk_small.show_batch(4, figsize=(12,12))

data_utk_big.show_batch(4, figsize=(12,12))

data_appa_small.show_batch(4, figsize=(12,12))

data_appa_big.show_batch(4, figsize=(12,12))

Fastai Modelling

import torch
import torch.nn as nn
import torch.nn.functional as F
from pathlib import Path
import random

opt_func = partial(optim.Adam, betas=(0.9,0.99), eps=1e-5)

def resnet(pretrained=False):
    pretrained = 'imagenet' if pretrained else None
    model = pretrainedmodels.resnet34(pretrained=pretrained)
    return model

class L1LossFlat(nn.SmoothL1Loss):
    def forward(self, input:Tensor, target:Tensor) -> Rank0Tensor:
        return super().forward(input.view(-1), target.view(-1))

df_utk_small = data_utk_small.to_df()

df_utk_small.head()

df_utk_small.rename(columns = {"x":"full_path", "y":"age"}, inplace=True)
df_utk_small['age'] = df_utk_small['age'].apply(lambda x: int(float(x)))

df_utk_small.head()

df_appa.head()

src_wiki = '../input/wiki-face-data/wiki_crop/wiki_crop/'
src_utk = '../input/utk-face-cropped/utkcropped/utkcropped/'
src_appa = '../input/appa-real-face-cropped/final_files/'

dest = '../working/wiki_utk/'

import os
import shutil
import pathlib

pathlib.Path(dest).mkdir(parents=True, exist_ok=True)

os.listdir(dest)

[]

for root, dirs, files in os.walk(src_wiki):
    for file in files:
        path_file = os.path.join(root,file)
        shutil.copy2(path_file, dest)

for root, dirs, files in os.walk(src_utk):
    for file in files:
        path_file = os.path.join(root,file)
        shutil.copy2(path_file,dest)

for root, dirs, files in os.walk(src_appa):
    for file in files:
        path_file = os.path.join(root,file)
        shutil.copy2(path_file,dest)

df_age_wiki.head()

df_age_wiki['full_path'] = df_age_wiki['full_path'].str[3:]
df_age_wiki.head()

frames = [df_age_wiki, df_utk_small, df_appa]
df_wiki_utk_appa = pd.concat(frames)

df_wiki_utk_appa.head()

df_wiki_utk_appa.shape

(43655, 2)

os.listdir(dest)

['29_1_1_20170112234606137.jpg.chip.jpg',
 '48_1_0_20170105000634011.jpg.chip.jpg',
 '32_0_1_20170113001322715.jpg.chip.jpg',
 '26_0_0_20170116192554607.jpg.chip.jpg',
 '36561562_1888-07-24_2004.jpg',
 '2338017_1960-09-21_2002.jpg',
 '3484443_1981-06-17_2006.jpg',
 '35_1_1_20170113003536772.jpg.chip.jpg',
 '52517_1934-10-07_1964.jpg',
 '46348054_1920-08-10_1942.jpg',
 '15_1_2_20170102234824195.jpg.chip.jpg',
 '903342_1972-06-28_2008.jpg',
 '29_0_1_20170117094745480.jpg.chip.jpg',
 '3762844_1945-09-25_1978.jpg',
 '27846664_1901-11-01_2010.jpg',
 '1051405_1969-11-19_2007.jpg',
 '249274_1955-07-01_2012.jpg',
 '20053572_1982-04-06_2009.jpg',
 '24815184_1864-08-15_2004.jpg',
 '001081.jpg',
 '12653807_1970-07-21_2011.jpg',
 '1_1_0_20170109193704430.jpg.chip.jpg',
 '26537728_1951-10-11_1978.jpg',
 '9952833_1990-06-05_2013.jpg',
 '88_0_1_20170116144533236.jpg.chip.jpg',
 '9632534_1957-06-09_2000.jpg',
 '36_1_0_20170104172916913.jpg.chip.jpg',
 '7796_1884-06-21_1941.jpg',
 '39160244_1989-03-05_2012.jpg',
 '50_1_0_20170117092638126.jpg.chip.jpg',
 '29_1_0_20170104172655682.jpg.chip.jpg',
 '7118448_1972-09-17_2008.jpg',
 '38372732_1939-06-09_1960.jpg',
 '35990083_1960-11-24_2001.jpg',
 '40_1_1_20170112231504476.jpg.chip.jpg',
 '27_1_2_20170116170956198.jpg.chip.jpg',
 '38_0_0_20170117175858084.jpg.chip.jpg',
 '28449525_1950-03-17_1994.jpg',
 '682290_1951-07-24_2009.jpg',
 '30079345_1993-07-20_2014.jpg',
 '46776424_1890-12-19_1944.jpg',
 '24_1_1_20170112204751139.jpg.chip.jpg',
 '26_0_3_20170119181310597.jpg.chip.jpg',
 '198400_1913-02-19_1961.jpg',
 '29200196_1970-01-01_2014.jpg',
 '25842166_1987-05-23_2011.jpg',
 '24576620_1956-08-20_2010.jpg',
 '000244.jpg',
 '46243758_1925-03-26_1990.jpg',
 '18767507_1931-05-01_2008.jpg',
 '99_1_0_20170120134718759.jpg.chip.jpg',
 '12970907_1930-10-16_1948.jpg',
 '001372.jpg',
 '378848_1945-08-03_2013.jpg',
 '20751442_1988-03-02_2012.jpg',
 '2446664_1978-10-26_2008.jpg',
 '8869997_1981-02-13_2012.jpg',
 '32504267_1954-04-17_1979.jpg',
 '19186371_1985-06-10_2009.jpg',
 '34430247_1896-12-30_1944.jpg',
 '27_1_1_20170113001031147.jpg.chip.jpg',
 '1_0_3_20161220143121101.jpg.chip.jpg',
 '501287_1955-05-10_2007.jpg',
 '65_0_0_20170111193809606.jpg.chip.jpg',
 '46596624_1960-02-16_1979.jpg',
 '14950334_1971-05-26_2006.jpg',
 '18418711_1904-10-16_1952.jpg',
 '5853885_1969-08-08_2014.jpg',
 '36836003_1964-12-19_1990.jpg',
 '853348_1931-10-26_2012.jpg',
 '40_0_0_20170117181229922.jpg.chip.jpg',
 '5_1_4_20161221201952681.jpg.chip.jpg',
 '4877949_1983-10-27_2013.jpg',
 '1662630_1971-05-01_2005.jpg',
 '4498970_1994-02-27_2013.jpg',
 '24_1_2_20170116171627787.jpg.chip.jpg',
 '15187291_1901-05-31_1943.jpg',
 '1096879_1945-09-06_2005.jpg',
 '39_0_3_20170104220721736.jpg.chip.jpg',
 '28707402_1988-12-30_2012.jpg',
 '1_0_0_20170109191453449.jpg.chip.jpg',
 '42361164_1995-12-14_2014.jpg',
 '34564786_1961-09-13_2007.jpg',
 '21_1_1_20170114032154585.jpg.chip.jpg',
 '28120539_1989-06-25_2009.jpg',
 '1274866_1939-01-06_1985.jpg',
 '25_1_0_20170117195346437.jpg.chip.jpg',
 '24353667_1924-04-01_1950.jpg',
 '003526.jpg',
 '24086813_1923-05-13_1943.jpg',
 '2066719_1983-06-20_2013.jpg',
 '32_0_1_20170116215755940.jpg.chip.jpg',
 '006293.jpg',
 '006260.jpg',
 '603566_1940-07-18_2012.jpg',
 '950281_1967-10-01_2004.jpg',
 '30_1_1_20170114024643760.jpg.chip.jpg',
 '003307.jpg',
 '000010.jpg',
 '58_0_0_20170113185223064.jpg.chip.jpg',
 '5340721_1945-08-19_2013.jpg',
 '4462806_1980-11-13_2011.jpg',
 '21750900_1984-06-06_2009.jpg',
 '1_0_2_20161219221816327.jpg.chip.jpg',
 '15685887_1942-09-11_1966.jpg',
 '9236375_1971-09-02_2010.jpg',
 '17_1_3_20170104221735351.jpg.chip.jpg',
 '15823114_1954-10-06_2006.jpg',
 '1573777_1943-12-31_2005.jpg',
 '3007277_1954-09-07_2013.jpg',
 '38479138_1986-04-23_2006.jpg',
 '006755.jpg',
 '1480772_1942-09-26_1990.jpg',
 '42244963_1948-04-19_2014.jpg',
 '51_0_0_20170120222842138.jpg.chip.jpg',
 '36422331_1982-02-28_2012.jpg',
 '7895818_1927-04-06_1960.jpg',
 '44744914_1941-02-22_2007.jpg',
 '796076_1969-05-16_2010.jpg',
 '13106750_1986-09-27_2011.jpg',
 '24_1_2_20170116165633882.jpg.chip.jpg',
 '61_1_2_20170105174916768.jpg.chip.jpg',
 '1567044_1969-10-24_2009.jpg',
 '3614080_1965-08-10_2013.jpg',
 '11775419_1978-07-26_2008.jpg',
 '58_0_3_20170119211559343.jpg.chip.jpg',
 '21178468_1951-12-20_1973.jpg',
 '004508.jpg',
 '006975.jpg',
 '40800834_1971-01-31_2010.jpg',
 '55_0_0_20170117185729852.jpg.chip.jpg',
 '37201426_1985-02-03_2014.jpg',
 '26788886_1962-10-14_1984.jpg',
 '37_0_0_20170109013120280.jpg.chip.jpg',
 '1_1_2_20161219201453804.jpg.chip.jpg',
 '39611913_1993-02-15_2014.jpg',
 '38379760_1990-05-14_2013.jpg',
 '29351931_1927-01-20_1990.jpg',
 '30_1_0_20170116201303368.jpg.chip.jpg',
 '10016074_1987-07-27_2012.jpg',
 '24279039_1981-04-23_2008.jpg',
 '30687645_1992-05-29_2011.jpg',
 '44285586_1992-06-19_2012.jpg',
 '51_0_0_20170111171747340.jpg.chip.jpg',
 '16534354_1989-08-21_2012.jpg',
 '28_0_3_20170119194803955.jpg.chip.jpg',
 '37128181_1992-01-13_2012.jpg',
 '34686067_1921-09-09_1960.jpg',
 '45422904_1996-03-05_2015.jpg',
 '37126092_1991-07-06_2014.jpg',
 '42520670_1940-02-11_1960.jpg',
 '89_0_1_20170117193440263.jpg.chip.jpg',
 '37_0_4_20170104000748917.jpg.chip.jpg',
 '43619465_1989-03-22_2014.jpg',
 '004828.jpg',
 '004922.jpg',
 '1624907_1927-08-07_2007.jpg',
 '20016339_1992-02-09_2009.jpg',
 '29_0_1_20170108235740510.jpg.chip.jpg',
 '25_1_1_20170112205103738.jpg.chip.jpg',
 '31_1_3_20170109134554005.jpg.chip.jpg',
 '779097_1977-03-06_2008.jpg',
 '25_0_2_20170117195218570.jpg.chip.jpg',
 '8027145_1978-07-25_1998.jpg',
 '2443611_1976-05-25_2007.jpg',
 '87_0_1_20170117194805270.jpg.chip.jpg',
 '2026045_1955-05-23_2007.jpg',
 '15803680_1942-12-26_2008.jpg',
 '1712978_1977-12-10_2008.jpg',
 '21644506_1985-07-05_2011.jpg',
 '6733489_1944-11-02_1968.jpg',
 '32432553_1922-02-26_1947.jpg',
 '000381.jpg',
 '3354295_1995-07-09_2008.jpg',
 '40890850_1949-07-20_2013.jpg',
 '2872703_1965-09-24_2012.jpg',
 '19447796_1989-07-26_2012.jpg',
 '10977057_1980-11-24_2008.jpg',
 '6145763_1960-02-19_2008.jpg',
 '35584490_1990-01-31_2012.jpg',
 '204942_1955-07-14_2011.jpg',
 '33947707_1995-12-29_2013.jpg',
 '28285893_1992-02-25_2011.jpg',
 '4_0_0_20170110213530901.jpg.chip.jpg',
 '2507966_1973-08-03_2014.jpg',
 '005261.jpg',
 '32_1_0_20170117174838232.jpg.chip.jpg',
 '1706281_1942-06-16_2003.jpg',
 '16617089_1963-03-09_2009.jpg',
 '39211197_1943-02-24_2006.jpg',
 '30501540_1960-07-29_1990.jpg',
 '25854312_1968-05-02_2010.jpg',
 '45_0_3_20170119203905073.jpg.chip.jpg',
 '14808073_1952-08-25_2015.jpg',
 '002745.jpg',
 '1633417_1912-02-20_1982.jpg',
 '31789140_1980-02-25_2000.jpg',
 '35_1_1_20170113005402981.jpg.chip.jpg',
 '2802601_1927-03-13_2002.jpg',
 '29462622_1935-06-10_1960.jpg',
 '18115101_1946-06-18_1941.jpg',
 '42_0_1_20170117171503046.jpg.chip.jpg',
 '30878743_1977-11-29_2000.jpg',
 '1234991_1912-03-22_1952.jpg',
 '006942.jpg',
 '19487892_1983-10-11_2015.jpg',
 '9427912_1969-10-23_2006.jpg',
 '36_0_0_20170109003634534.jpg.chip.jpg',
 '23617680_1929-09-28_1971.jpg',
 '7976847_1985-07-30_2011.jpg',
 '42822079_1970-11-30_2013.jpg',
 '79_0_0_20170111205953011.jpg.chip.jpg',
 '37732462_1993-07-28_2012.jpg',
 '58_0_0_20170117191839091.jpg.chip.jpg',
 '30_0_1_20170113155013364.jpg.chip.jpg',
 '1262985_1932-05-19_1963.jpg',
 '750603_1942-10-21_2012.jpg',
 '18600259_1982-02-28_2010.jpg',
 '29582036_1988-06-21_2008.jpg',
 '2_1_3_20161220144844310.jpg.chip.jpg',
 '28_1_0_20170117092254042.jpg.chip.jpg',
 '36967227_1897-12-02_2008.jpg',
 '1_0_3_20161219225759904.jpg.chip.jpg',
 '52_0_0_20170104204156747.jpg.chip.jpg',
 '46498373_1938-01-26_1959.jpg',
 '5197788_1959-12-31_2005.jpg',
 '14307202_1985-01-16_2007.jpg',
 '13498620_1985-06-25_2014.jpg',
 '1467908_1927-01-01_1948.jpg',
 '6777493_1985-01-09_2013.jpg',
 '38842697_1993-08-09_2014.jpg',
 '17264219_1933-04-03_1956.jpg',
 '7725612_1974-07-07_2011.jpg',
 '002618.jpg',
 '4_0_0_20170110213253207.jpg.chip.jpg',
 '30268663_1970-03-01_2013.jpg',
 '29797248_1989-10-22_2013.jpg',
 '32_1_3_20170119155703795.jpg.chip.jpg',
 '40519566_1941-01-10_1975.jpg',
 '004994.jpg',
 '28790298_1988-03-20_2011.jpg',
 '32888889_1976-12-15_2014.jpg',
 '13069966_1959-05-09_2006.jpg',
 '003129.jpg',
 '201160_1914-09-15_1968.jpg',
 '8347154_1935-12-09_1955.jpg',
 '28732281_1986-09-24_2010.jpg',
 '21346551_1931-01-09_1958.jpg',
 '2486630_1948-01-27_2006.jpg',
 '476099_1946-02-05_2009.jpg',
 '8006688_1961-03-14_2007.jpg',
 '4442797_1971-09-16_2014.jpg',
 '29950756_1986-02-28_2015.jpg',
 '16_0_0_20170110231218369.jpg.chip.jpg',
 '11496750_1930-05-19_1986.jpg',
 '44681795_1984-09-13_2015.jpg',
 '3591776_1946-12-31_1972.jpg',
 '3_1_4_20161221193144389.jpg.chip.jpg',
 '1984786_1930-11-23_2004.jpg',
 '4065567_1942-08-26_1998.jpg',
 '39535197_1988-11-28_2014.jpg',
 '1982061_1975-03-15_2012.jpg',
 '2593429_1947-09-19_1976.jpg',
 '769156_1882-02-04_1944.jpg',
 '45555747_1956-06-03_2015.jpg',
 '88833_1916-07-01_1945.jpg',
 '38032109_1992-04-18_2015.jpg',
 '002553.jpg',
 '7052971_1978-08-15_2010.jpg',
 '30760593_1946-02-05_1968.jpg',
 '26_0_1_20170113135516970.jpg.chip.jpg',
 '870163_1899-09-22_1948.jpg',
 '004412.jpg',
 '6056366_1954-11-11_1950.jpg',
 '28624328_1953-08-23_1973.jpg',
 '55_0_0_20170120221856526.jpg.chip.jpg',
 '1607852_1976-08-06_2007.jpg',
 '21_1_4_20170103223204087.jpg.chip.jpg',
 '10982693_1955-12-10_1992.jpg',
 '17958040_1987-07-24_2011.jpg',
 '20670994_1961-05-03_1988.jpg',
 '32216891_1935-05-24_1960.jpg',
 '67291_1937-03-22_1960.jpg',
 '50_0_0_20170117153725428.jpg.chip.jpg',
 '44_1_1_20170117142913571.jpg.chip.jpg',
 '004467.jpg',
 '24_1_0_20170117150535820.jpg.chip.jpg',
 '1_0_2_20161219151026723.jpg.chip.jpg',
 '9_0_2_20170110224822473.jpg.chip.jpg',
 '40_0_0_20170116221022897.jpg.chip.jpg',
 '35798200_1987-08-03_2014.jpg',
 '002576.jpg',
 '222737_1955-10-12_2007.jpg',
 '25644669_1953-10-28_2009.jpg',
 '45_0_0_20170116212320846.jpg.chip.jpg',
 '5841825_1985-03-15_2012.jpg',
 '60_0_0_20170120223043268.jpg.chip.jpg',
 '46340639_1945-04-03_1969.jpg',
 '17256613_1950-04-21_1984.jpg',
 '20050887_1973-07-21_1996.jpg',
 '005593.jpg',
 '46730387_1922-05-27_2013.jpg',
 '33415719_1937-03-23_1962.jpg',
 '24_1_0_20170117150310308.jpg.chip.jpg',
 '719076_1960-09-04_2012.jpg',
 '14146965_1983-11-27_2013.jpg',
 '7391254_1936-08-21_1962.jpg',
 '38594467_1992-03-08_2014.jpg',
 '32222137_1984-11-24_2010.jpg',
 '27_1_1_20170116214220570.jpg.chip.jpg',
 '10046706_1933-04-21_1962.jpg',
 '000005.jpg',
 '32_1_0_20170103180822745.jpg.chip.jpg',
 '30049841_1969-11-04_2015.jpg',
 '429097_1954-06-02_2007.jpg',
 '85_0_0_20170111210559755.jpg.chip.jpg',
 '001082.jpg',
 '3062692_1965-07-31_2011.jpg',
 '52_0_0_20170117135533022.jpg.chip.jpg',
 '32227332_1992-08-24_2015.jpg',
 '1_0_2_20161219204958037.jpg.chip.jpg',
 '1809170_1967-08-01_2011.jpg',
 '20816070_1971-05-10_2007.jpg',
 '35_1_0_20170103182732026.jpg.chip.jpg',
 '27637550_1917-03-10_2007.jpg',
 '15_0_0_20170104013333801.jpg.chip.jpg',
 '39961436_1925-01-26_1948.jpg',
 '21291667_1913-10-02_1950.jpg',
 '006552.jpg',
 '40492912_1988-06-30_2013.jpg',
 '42451669_1983-04-12_2012.jpg',
 '329524_1945-10-24_2014.jpg',
 '24_1_1_20170112192919494.jpg.chip.jpg',
 '8_0_0_20170110220118138.jpg.chip.jpg',
 '38416948_1962-07-13_1991.jpg',
 '33791098_1979-10-04_2011.jpg',
 '28_0_1_20170113142308413.jpg.chip.jpg',
 '40210503_1992-08-30_2014.jpg',
 '9509458_1993-01-31_2008.jpg',
 '39098401_1921-10-29_1952.jpg',
 '23_1_0_20170117144740562.jpg.chip.jpg',
 '42_1_2_20170116162349143.jpg.chip.jpg',
 '41978653_1951-07-27_2008.jpg',
 '88371_1944-05-20_2013.jpg',
 '44_1_3_20170119172158321.jpg.chip.jpg',
 '20329296_1983-10-13_2008.jpg',
 '001972.jpg',
 '31976668_1991-03-02_2015.jpg',
 '007118.jpg',
 '32752294_1973-02-07_1997.jpg',
 '50_1_0_20170117155139407.jpg.chip.jpg',
 '002627.jpg',
 '39220677_1976-04-22_1998.jpg',
 '26_0_3_20170119180948868.jpg.chip.jpg',
 '5101720_1975-06-30_2014.jpg',
 '34686543_1948-01-09_1992.jpg',
 '14276461_1983-04-18_2011.jpg',
 '5445094_1936-08-13_2012.jpg',
 '7_0_2_20161219193457075.jpg.chip.jpg',
 '1_1_2_20161219162243182.jpg.chip.jpg',
 '3060227_1974-10-02_2012.jpg',
 '39903017_1989-08-26_2013.jpg',
 '27962750_1894-03-20_1946.jpg',
 '42149298_1978-09-03_2014.jpg',
 '006396.jpg',
 '27_0_0_20170116204136393.jpg.chip.jpg',
 '10007577_1988-06-16_2007.jpg',
 '44835846_1946-12-08_1978.jpg',
 '26_1_1_20170116162111804.jpg.chip.jpg',
 '13713930_1976-06-27_2012.jpg',
 '27_1_2_20170116163622032.jpg.chip.jpg',
 '004115.jpg',
 '27_1_3_20170109132029072.jpg.chip.jpg',
 '003751.jpg',
 '9201851_1923-10-10_1953.jpg',
 '27_1_4_20170103235051403.jpg.chip.jpg',
 '13551558_1980-09-17_2010.jpg',
 '15483779_1950-05-02_1950.jpg',
 '1057264_1947-03-06_1974.jpg',
 '001200.jpg',
 '32_1_1_20170112212952111.jpg.chip.jpg',
 '41942456_1987-06-18_2014.jpg',
 '40_1_0_20170117160718607.jpg.chip.jpg',
 '3538953_1967-05-14_2013.jpg',
 '5928640_1957-06-14_2013.jpg',
 '18933035_1982-08-26_2012.jpg',
 '27_1_0_20170117120642103.jpg.chip.jpg',
 '37_0_1_20170117134417970.jpg.chip.jpg',
 '42_1_0_20170117121915102.jpg.chip.jpg',
 '3603119_1940-08-28_2011.jpg',
 '4163901_1950-08-20_1974.jpg',
 '23814332_1990-02-21_2007.jpg',
 '35_1_0_20170117170314803.jpg.chip.jpg',
 '35_0_3_20170104214246948.jpg.chip.jpg',
 '2415032_1954-02-19_2012.jpg',
 '8277549_1979-11-06_2005.jpg',
 '44117991_1918-02-14_1944.jpg',
 '42_0_0_20170104184335702.jpg.chip.jpg',
 '004633.jpg',
 '27857682_1916-11-13_1970.jpg',
 '14475885_1953-02-25_2010.jpg',
 '17_0_0_20170114025913647.jpg.chip.jpg',
 '36_1_1_20170113011506296.jpg.chip.jpg',
 '297985_1911-07-16_1955.jpg',
 '44400523_1928-07-26_1971.jpg',
 '35_1_0_20170116222743531.jpg.chip.jpg',
 '28_0_2_20170116191256645.jpg.chip.jpg',
 '40773413_1985-05-18_2013.jpg',
 '918149_1890-07-11_1943.jpg',
 '7218188_1967-05-04_2012.jpg',
 '81_0_1_20170117193731174.jpg.chip.jpg',
 '70_0_0_20170117160355895.jpg.chip.jpg',
 '6215810_1973-11-27_2008.jpg',
 '18_1_0_20170109213856511.jpg.chip.jpg',
 '55_0_0_20170120223706509.jpg.chip.jpg',
 '003723.jpg',
 '3569434_1980-10-01_2012.jpg',
 '25_1_0_20170117193539276.jpg.chip.jpg',
 '19216583_1937-05-10_1964.jpg',
 '12636288_1975-11-30_2008.jpg',
 '41347756_1991-03-10_2012.jpg',
 '12417728_1949-02-02_1980.jpg',
 '37684007_1945-05-07_2012.jpg',
 '004260.jpg',
 '16_1_0_20170109213608540.jpg.chip.jpg',
 '44695408_1911-02-24_1991.jpg',
 '1639687_1967-07-14_1996.jpg',
 '19345924_1988-09-13_2008.jpg',
 '2062331_1976-06-18_2013.jpg',
 '3628240_1960-12-17_1987.jpg',
 '10529976_1930-05-20_1954.jpg',
 '27_0_1_20170117175753874.jpg.chip.jpg',
 '46_0_3_20170119204802431.jpg.chip.jpg',
 '41470713_1952-12-26_1974.jpg',
 '180733_1935-05-08_1969.jpg',
 '18900914_1985-01-02_2006.jpg',
 '22421796_1955-06-25_1983.jpg',
 '71_1_0_20170110140328613.jpg.chip.jpg',
 '2476333_1981-06-01_2010.jpg',
 '17_1_0_20170109212920120.jpg.chip.jpg',
 '1167372_1903-08-14_1952.jpg',
 '16555375_1988-11-12_2015.jpg',
 '2323881_1971-03-05_2007.jpg',
 '2_0_2_20161219141647272.jpg.chip.jpg',
 '16_0_0_20170110232332618.jpg.chip.jpg',
 '40826007_1996-05-31_2014.jpg',
 '27091124_1988-03-14_2011.jpg',
 '41_0_1_20170117173648607.jpg.chip.jpg',
 '41_0_4_20170104205417148.jpg.chip.jpg',
 '36242663_1992-04-13_2012.jpg',
 '32999146_1949-07-22_2011.jpg',
 '28509106_1927-11-12_2009.jpg',
 '24994_1956-09-23_2014.jpg',
 '20865712_1853-09-05_2004.jpg',
 '26_0_1_20170117154704178.jpg.chip.jpg',
 '258756_1951-07-31_1971.jpg',
 '378171_1941-12-01_2011.jpg',
 '32942324_1927-12-13_1951.jpg',
 '5767123_1984-12-20_2015.jpg',
 '23167046_1987-08-01_2015.jpg',
 '23560198_1992-06-20_2014.jpg',
 '2_1_3_20161220145026015.jpg.chip.jpg',
 '004750.jpg',
 '43730875_1991-07-25_2014.jpg',
 '36245423_1917-12-27_1985.jpg',
 '39440583_1906-10-16_1944.jpg',
 '001140.jpg',
 '35414147_1993-12-24_2012.jpg',
 '37763795_1946-06-22_2009.jpg',
 '5482429_1956-05-15_1978.jpg',
 '11189557_1943-10-02_2011.jpg',
 '6786417_1934-01-29_1972.jpg',
 '87637_1961-09-26_2013.jpg',
 '1088551_1934-10-01_1961.jpg',
 '560012_1954-12-31_2009.jpg',
 '36890678_1955-04-19_1970.jpg',
 '29_1_2_20170116173225104.jpg.chip.jpg',
 '32466540_1964-11-09_2013.jpg',
 '43663635_1988-05-12_2012.jpg',
 '3329342_1982-02-15_2008.jpg',
 '2057979_1959-02-05_2013.jpg',
 '9612190_1986-03-05_2009.jpg',
 '537999_1974-01-14_2005.jpg',
 '1_1_4_20161221193744550.jpg.chip.jpg',
 '34_1_2_20170116193448303.jpg.chip.jpg',
 '001413.jpg',
 '31_1_1_20170116155750618.jpg.chip.jpg',
 '4095083_1987-04-26_2015.jpg',
 '6511626_1984-11-25_2011.jpg',
 '26_0_1_20170113134739602.jpg.chip.jpg',
 '2_0_0_20161219162303390.jpg.chip.jpg',
 '432575_1979-07-03_2013.jpg',
 '505670_1941-10-25_1974.jpg',
 '7144873_1987-04-12_2013.jpg',
 '18249084_1985-06-19_2012.jpg',
 '23864942_1981-05-31_2010.jpg',
 '12501_1941-12-08_2008.jpg',
 '696094_1963-04-18_2012.jpg',
 '003933.jpg',
 '45131582_1932-07-01_1954.jpg',
 '2275438_1977-09-28_2005.jpg',
 '35724503_1948-09-19_1966.jpg',
 '12878785_1989-04-30_2011.jpg',
 '34_0_3_20170119165846359.jpg.chip.jpg',
 '1266534_1975-11-03_2011.jpg',
 '16354477_1959-07-15_2008.jpg',
 '001617.jpg',
 '28_1_0_20170113001234883.jpg.chip.jpg',
 '11696343_1989-03-03_2009.jpg',
 '46518299_1923-05-28_1947.jpg',
 '961110_1959-09-14_2002.jpg',
 '20128467_1937-04-23_2008.jpg',
 '24476862_1989-05-11_2015.jpg',
 '5948348_1987-09-17_2007.jpg',
 '26_0_0_20170117172201507.jpg.chip.jpg',
 '30_0_0_20170119145340944.jpg.chip.jpg',
 '1_0_2_20161219162852759.jpg.chip.jpg',
 '13283895_1976-01-10_2000.jpg',
 '1230336_1955-09-25_2007.jpg',
 '5302234_1939-10-16_1971.jpg',
 '41142422_1986-11-13_2014.jpg',
 '1_0_0_20170110213012671.jpg.chip.jpg',
 '5376225_1974-03-31_2009.jpg',
 '31730021_1984-11-09_2011.jpg',
 '27_1_1_20170117120629605.jpg.chip.jpg',
 '21633828_1984-12-05_2013.jpg',
 '35_0_3_20170119183157029.jpg.chip.jpg',
 '24_1_0_20170117150848786.jpg.chip.jpg',
 '3970134_1987-03-17_1980.jpg',
 '3078676_1944-01-29_2008.jpg',
 '11210768_1922-02-13_1948.jpg',
 '22316377_1989-06-01_2012.jpg',
 '37_0_1_20170116214356844.jpg.chip.jpg',
 '1145996_1948-07-22_2007.jpg',
 '26765106_1986-01-08_2014.jpg',
 '65_0_2_20170117000746182.jpg.chip.jpg',
 '36753693_1936-11-26_1965.jpg',
 '41707776_1991-10-11_2014.jpg',
 '4148260_1989-05-17_2010.jpg',
 '35_0_0_20170117134955604.jpg.chip.jpg',
 '1254277_1984-04-18_2010.jpg',
 '22892544_1923-11-01_2009.jpg',
 '2058760_1915-12-02_2012.jpg',
 '46471365_1935-10-08_1960.jpg',
 '29_1_0_20170111182452813.jpg.chip.jpg',
 '70_1_3_20170119150943854.jpg.chip.jpg',
 '25_0_4_20170117151414412.jpg.chip.jpg',
 '21741673_1995-03-01_2009.jpg',
 '148878_1965-10-05_2005.jpg',
 '13392046_1929-06-05_2002.jpg',
 '28682121_1988-04-28_2011.jpg',
 '33305711_1987-07-12_2012.jpg',
 '25_1_3_20170117152019467.jpg.chip.jpg',
 '41388906_1951-04-03_1974.jpg',
 '36529968_1979-10-25_2011.jpg',
 '45522223_1902-05-19_1944.jpg',
 '15699247_1942-07-07_1978.jpg',
 '70_0_3_20170117191256001.jpg.chip.jpg',
 '171517_1939-04-21_2004.jpg',
 '49_1_0_20170104235723082.jpg.chip.jpg',
 '45_0_4_20170117202958949.jpg.chip.jpg',
 '36348127_1990-03-23_2012.jpg',
 '26_0_2_20170116181346232.jpg.chip.jpg',
 '43223_1941-12-31_2006.jpg',
 '45376382_1996-10-30_2015.jpg',
 '7999955_1950-11-20_2011.jpg',
 '8978138_1915-01-22_1975.jpg',
 '55_0_0_20170117191627092.jpg.chip.jpg',
 '10123489_1984-01-26_2009.jpg',
 '890_1981-06-07_2009.jpg',
 '22104041_1987-05-12_2011.jpg',
 '3964439_1987-01-21_2009.jpg',
 '4054196_1974-01-04_2005.jpg',
 '5562149_1979-03-06_2006.jpg',
 '600233_1961-06-27_2013.jpg',
 '17145298_1984-11-18_2011.jpg',
 '2102231_1947-04-18_2008.jpg',
 '31_1_0_20170117133121639.jpg.chip.jpg',
 '1631693_1913-10-27_2009.jpg',
 '670462_1984-09-07_2011.jpg',
 '11253737_1918-08-11_1947.jpg',
 '30_0_1_20170117202813031.jpg.chip.jpg',
 '29485869_1990-06-07_2012.jpg',
 '29841300_1985-04-25_1980.jpg',
 '6944155_1971-06-05_2012.jpg',
 '41162605_1990-09-09_1990.jpg',
 '5771217_1989-01-17_2013.jpg',
 '3280564_1981-11-26_2014.jpg',
 '34123747_1966-03-12_2011.jpg',
 '001552.jpg',
 '23_0_3_20170119164205782.jpg.chip.jpg',
 '37104937_1995-02-13_2013.jpg',
 '58_0_3_20170119195320690.jpg.chip.jpg',
 '2697684_1916-01-19_2011.jpg',
 '7705333_1984-09-29_2009.jpg',
 '25709268_1943-03-10_2015.jpg',
 '26_0_3_20170119162753492.jpg.chip.jpg',
 '2108919_1973-10-26_2014.jpg',
 '4262860_1926-11-09_2010.jpg',
 '1917633_1964-04-04_1973.jpg',
 '26_1_3_20170117144854965.jpg.chip.jpg',
 '2068375_1964-08-25_2014.jpg',
 '12_1_4_20170109214232071.jpg.chip.jpg',
 '41490857_1964-07-01_1986.jpg',
 '33701731_1955-08-16_2002.jpg',
 '1967356_1968-09-02_2007.jpg',
 '55_0_0_20170116193520434.jpg.chip.jpg',
 '1180042_1930-04-03_1961.jpg',
 '1258483_1981-05-04_2008.jpg',
 '27420958_1990-03-29_2015.jpg',
 '26531370_1963-10-06_2012.jpg',
 '1783180_2000-10-31_2011.jpg',
 '35_0_1_20170113141808954.jpg.chip.jpg',
 '37_1_0_20170117140111378.jpg.chip.jpg',
 '31305274_1957-06-02_2003.jpg',
 '21005625_1918-01-13_1945.jpg',
 '6743730_1921-03-28_1984.jpg',
 '8920426_1983-05-31_2011.jpg',
 '43853176_1989-09-12_2013.jpg',
 '38_1_0_20170117154128620.jpg.chip.jpg',
 '16_1_4_20170103214347949.jpg.chip.jpg',
 '88_0_2_20170112224251748.jpg.chip.jpg',
 '26970398_1968-01-02_2011.jpg',
 '29_0_0_20170116192401102.jpg.chip.jpg',
 '1050883_1942-03-16_2002.jpg',
 '3806612_1950-10-08_1966.jpg',
 '46725990_1923-08-01_1948.jpg',
 '177180_1899-11-22_1947.jpg',
 '26_1_3_20170119193116354.jpg.chip.jpg',
 '003313.jpg',
 '4098246_1978-05-31_2005.jpg',
 '006636.jpg',
 '5977658_1935-03-19_1964.jpg',
 '26457252_1980-11-08_2013.jpg',
 '26_0_2_20170104022626733.jpg.chip.jpg',
 '28_1_2_20170116162435106.jpg.chip.jpg',
 '45054492_1979-12-30_2014.jpg',
 '34904258_1977-07-10_2014.jpg',
 '30974716_1986-10-12_2014.jpg',
 '24720665_1989-04-28_2015.jpg',
 '38385843_1991-01-12_2013.jpg',
 '65_1_0_20170110143236073.jpg.chip.jpg',
 '23253790_1894-12-05_2008.jpg',
 '1134432_1993-01-26_2011.jpg',
 '10110600_1985-09-17_2012.jpg',
 '32_1_0_20170108224936101.jpg.chip.jpg',
 '39794722_1922-09-30_1958.jpg',
 '1267629_1947-10-04_2010.jpg',
 '34411645_1985-04-23_2011.jpg',
 '20459229_1973-05-11_2011.jpg',
 '27_1_0_20170116220503099.jpg.chip.jpg',
 '39277041_1949-06-29_1995.jpg',
 '27_0_0_20170116232646294.jpg.chip.jpg',
 '82471_1961-03-16_2011.jpg',
 '46672958_1956-04-23_1978.jpg',
 '13184655_1990-06-21_2010.jpg',
 '233512_1972-07-07_2010.jpg',
 '001508.jpg',
 '35847193_1989-09-21_2014.jpg',
 '9976273_1960-08-15_1988.jpg',
 '32384638_1990-07-12_2009.jpg',
 '2121079_1981-08-26_2009.jpg',
 '10_0_0_20170110220654150.jpg.chip.jpg',
 '9838372_1906-09-01_2008.jpg',
 '284313_1943-04-24_2008.jpg',
 '28910941_1959-05-01_2008.jpg',
 '004375.jpg',
 '2404184_1926-04-17_1946.jpg',
 '31_0_1_20170116010634833.jpg.chip.jpg',
 '40081785_1966-10-27_1956.jpg',
 '24_0_1_20170113150729383.jpg.chip.jpg',
 '12_1_4_20161223230033380.jpg.chip.jpg',
 '8_1_0_20170109202926726.jpg.chip.jpg',
 '13073532_1934-08-09_2012.jpg',
 '000668.jpg',
 '7300200_1949-06-10_1940.jpg',
 '004515.jpg',
 '26_1_2_20170104022829221.jpg.chip.jpg',
 '8171044_1984-09-03_2009.jpg',
 '26_0_0_20170113210127395.jpg.chip.jpg',
 '473342_1957-06-14_2014.jpg',
 '22748865_1981-02-09_2004.jpg',
 '12677067_1979-01-24_2010.jpg',
 '1_1_0_20170109191129918.jpg.chip.jpg',
 '4415323_1983-05-24_2013.jpg',
 '50_0_3_20170119205556344.jpg.chip.jpg',
 '34827214_1976-02-16_1970.jpg',
 '26_1_0_20170117173613245.jpg.chip.jpg',
 '31436814_1983-01-29_2012.jpg',
 '36475873_1988-02-09_2012.jpg',
 '67749_1931-11-06_1970.jpg',
 '50_0_0_20170117135317385.jpg.chip.jpg',
 '004366.jpg',
 '9104418_1899-02-28_1942.jpg',
 '006172.jpg',
 '31028436_1979-11-02_2013.jpg',
 '2491752_1971-10-25_2004.jpg',
 '35061796_1964-12-07_2011.jpg',
 '39369330_1984-01-09_2008.jpg',
 '1_0_4_20161221201454961.jpg.chip.jpg',
 '44065241_1941-10-18_1966.jpg',
 '6_1_0_20170103212206588.jpg.chip.jpg',
 '21_1_3_20170104231613281.jpg.chip.jpg',
 '33_1_0_20170103181050000.jpg.chip.jpg',
 '45491923_1919-02-04_1946.jpg',
 '001847.jpg',
 '000171.jpg',
 '181232_1962-12-09_2007.jpg',
 '250278_1965-12-30_2006.jpg',
 '3984297_1956-05-12_2013.jpg',
 '26196169_1984-03-20_2010.jpg',
 '25660936_1963-06-06_2014.jpg',
 '38613357_1921-07-10_1961.jpg',
 '2821311_1947-12-30_1979.jpg',
 '32438826_1992-05-18_2013.jpg',
 '15422496_1988-04-28_2009.jpg',
 '24597672_1987-11-19_2010.jpg',
 '004507.jpg',
 '332134_1957-10-04_2009.jpg',
 '45569355_1995-10-29_2014.jpg',
 '26_1_0_20170117003252979.jpg.chip.jpg',
 '31712416_1980-06-19_2000.jpg',
 '23055210_1960-03-17_2013.jpg',
 '865964_1968-01-01_2009.jpg',
 '7623721_1978-03-01_2010.jpg',
 '38896201_2000-12-26_2014.jpg',
 '21699302_1978-09-02_2007.jpg',
 '72_0_0_20170111171747670.jpg.chip.jpg',
 '76_0_2_20170112204914106.jpg.chip.jpg',
 '39481001_1990-07-25_2012.jpg',
 '35_0_3_20170119212039049.jpg.chip.jpg',
 '40753749_1951-09-15_2008.jpg',
 '26_1_4_20170116220414012.jpg.chip.jpg',
 '2850239_1940-07-26_1974.jpg',
 '1944523_1976-01-09_2008.jpg',
 '30306491_1954-10-15_2014.jpg',
 '54_1_3_20170109142901150.jpg.chip.jpg',
 '6107450_1961-05-31_2010.jpg',
 '14410782_1925-02-08_1976.jpg',
 '36273685_1984-02-22_2013.jpg',
 '30_0_0_20170117181257163.jpg.chip.jpg',
 '218081_1918-02-01_1960.jpg',
 '7088375_1980-12-19_2006.jpg',
 '000679.jpg',
 '43399688_1994-08-03_2015.jpg',
 '3555882_1899-05-20_1940.jpg',
 '41365854_1980-06-19_2004.jpg',
 '16536131_1949-11-09_1980.jpg',
 '640889_1969-09-09_2008.jpg',
 '163892_1958-10-20_2012.jpg',
 '18888856_1989-03-19_2012.jpg',
 '31248425_1961-11-26_1990.jpg',
 '001744.jpg',
 '25918808_1982-06-22_2012.jpg',
 '529315_1906-01-14_1960.jpg',
 '28586654_1967-05-06_2007.jpg',
 '5468856_1976-04-07_2009.jpg',
 '26_1_0_20170117175317465.jpg.chip.jpg',
 '26_1_1_20170116231924481.jpg.chip.jpg',
 '34666103_1974-04-11_1996.jpg',
 '23_1_4_20170103233619053.jpg.chip.jpg',
 '26_1_3_20170119193148570.jpg.chip.jpg',
 '552779_1971-11-03_2006.jpg',
 '25_1_0_20170103180657288.jpg.chip.jpg',
 '161577_1980-06-08_2011.jpg',
 '26_1_1_20170116160509916.jpg.chip.jpg',
 '004715.jpg',
 '1925799_1967-01-30_2010.jpg',
 '207774_1925-06-09_1966.jpg',
 '23890218_1982-03-04_2009.jpg',
 '3313310_1965-04-28_2009.jpg',
 '4_1_3_20161219224933671.jpg.chip.jpg',
 '24179114_1938-10-20_2008.jpg',
 '571810_1956-09-17_2000.jpg',
 '651347_1955-04-06_2015.jpg',
 '005314.jpg',
 '15552802_1961-06-09_2009.jpg',
 '44108810_1926-06-28_1947.jpg',
 '34957545_1928-03-20_2013.jpg',
 '23877707_1926-07-23_1945.jpg',
 '327664_1951-01-12_1994.jpg',
 '36_0_0_20170113184113374.jpg.chip.jpg',
 '56_1_0_20170113182536919.jpg.chip.jpg',
 '8808513_1960-01-10_1983.jpg',
 '31621448_1991-04-04_2011.jpg',
 '3149083_1961-03-01_1990.jpg',
 '731994_1973-03-19_2007.jpg',
 '647373_1956-07-15_2010.jpg',
 '45396098_1976-01-10_2015.jpg',
 '1643259_1923-10-13_1961.jpg',
 '52_0_0_20170111201135454.jpg.chip.jpg',
 '195152_1931-11-26_2003.jpg',
 '23381415_1986-11-14_2011.jpg',
 '44119711_1819-11-17_2004.jpg',
 '252442_1896-08-15_1947.jpg',
 '1_0_4_20161219195147803.jpg.chip.jpg',
 '2377004_1972-01-30_2005.jpg',
 '32_1_0_20170116191618394.jpg.chip.jpg',
 '8087946_1944-04-21_1966.jpg',
 '13550616_1977-12-19_2010.jpg',
 '9322645_1971-08-26_2007.jpg',
 '10503610_1949-08-01_2005.jpg',
 '2385757_1961-05-08_2003.jpg',
 '2861958_1967-03-30_2011.jpg',
 '33983_1951-12-20_2000.jpg',
 '28_0_3_20170119194929915.jpg.chip.jpg',
 '524636_1961-04-08_2012.jpg',
 '30_1_0_20170117103601281.jpg.chip.jpg',
 '34876346_1946-05-13_2010.jpg',
 '34363325_1974-01-20_2001.jpg',
 '80_0_0_20170117175301125.jpg.chip.jpg',
 '28_0_0_20170116174356539.jpg.chip.jpg',
 '2_0_2_20161219202702788.jpg.chip.jpg',
 '22522134_1977-06-22_2002.jpg',
 '959464_1977-04-05_2004.jpg',
 '20597806_1980-10-04_2012.jpg',
 '28_0_0_20170116212102377.jpg.chip.jpg',
 '2134356_1966-03-14_2007.jpg',
 '36_0_3_20170119195506419.jpg.chip.jpg',
 '26_1_0_20170116164040161.jpg.chip.jpg',
 '5_0_4_20170103210509634.jpg.chip.jpg',
 '45681223_1993-06-25_2014.jpg',
 '43479903_1988-07-21_2014.jpg',
 '67030_1909-06-20_1940.jpg',
 '2857671_1920-02-14_1947.jpg',
 '36787370_1977-04-05_2010.jpg',
 '12167720_1984-09-30_2014.jpg',
 '10970703_1984-05-21_2011.jpg',
 '35022161_1984-11-13_2011.jpg',
 '45711961_1986-08-14_2008.jpg',
 '47_0_0_20170103183442514.jpg.chip.jpg',
 '3250661_1986-01-31_2009.jpg',
 '1896252_1942-09-02_2008.jpg',
 '5737964_1980-07-16_2007.jpg',
 '45277231_1926-02-26_1973.jpg',
 '40380141_1995-11-27_2013.jpg',
 '343294_1964-01-27_2001.jpg',
 '12754_1899-07-07_1946.jpg',
 '25648564_1936-12-07_1971.jpg',
 '7463952_1982-12-30_2013.jpg',
 '471630_1898-11-22_1978.jpg',
 '1148639_1963-12-26_2009.jpg',
 '40184108_1993-05-29_2013.jpg',
 '92331_1955-08-10_2008.jpg',
 '10016225_1988-02-03_2012.jpg',
 '40_0_2_20170116191354067.jpg.chip.jpg',
 '11700461_1983-10-11_2007.jpg',
 '32908796_1985-06-20_2013.jpg',
 '24_1_0_20170117191400699.jpg.chip.jpg',
 '24064016_1989-05-07_2008.jpg',
 '25211052_1991-07-08_2015.jpg',
 '23315533_1988-01-27_2011.jpg',
 '33999923_1993-04-30_2012.jpg',
 '79_1_0_20170120225803298.jpg.chip.jpg',
 '25615152_1987-11-11_2009.jpg',
 '11973758_1991-02-27_2008.jpg',
 '1246851_1974-08-09_2014.jpg',
 '26_0_2_20170116182347867.jpg.chip.jpg',
 '1235669_1976-09-11_2007.jpg',
 '2693444_1972-08-09_1965.jpg',
 '27_0_4_20170103235757172.jpg.chip.jpg',
 '30387921_1975-02-01_2010.jpg',
 '3331742_1969-03-26_2013.jpg',
 '17021305_1883-02-28_1967.jpg',
 '41691585_1995-09-25_2015.jpg',
 '3101416_1973-03-17_2009.jpg',
 '53_1_0_20170109220736654.jpg.chip.jpg',
 '34027685_1972-07-22_2005.jpg',
 '7514390_1977-05-25_2014.jpg',
 '3348502_1984-09-18_2007.jpg',
 '11021240_1887-08-31_1954.jpg',
 '1090974_1971-03-12_1988.jpg',
 '35_1_1_20170113012807626.jpg.chip.jpg',
 '913724_1979-04-03_2008.jpg',
 '34557038_1974-07-08_2011.jpg',
 '006995.jpg',
 '37_1_1_20170116232758932.jpg.chip.jpg',
 '5929080_1943-07-02_1997.jpg',
 '6171937_1981-05-29_2007.jpg',
 '2251257_1931-03-23_1964.jpg',
 '2776174_1969-02-20_2014.jpg',
 '65_1_0_20170103184138483.jpg.chip.jpg',
 '7856897_1986-05-23_2012.jpg',
 '46182062_1969-04-29_2013.jpg',
 '38128011_1979-08-14_2002.jpg',
 '1776092_1959-01-31_2007.jpg',
 '4647550_1949-10-31_2012.jpg',
 '2092862_1956-04-09_1984.jpg',
 '2201_1922-06-19_1955.jpg',
 '68_0_0_20170117172938504.jpg.chip.jpg',
 '33369667_1989-01-28_2012.jpg',
 '8118857_1953-06-11_1977.jpg',
 '28968145_1985-04-03_2007.jpg',
 '1768989_1953-07-21_2008.jpg',
 '849454_1955-04-17_2013.jpg',
 '26908645_1973-03-22_2015.jpg',
 '18109220_1987-08-04_2015.jpg',
 '8286385_1922-11-01_1953.jpg',
 '1405558_1968-03-14_2012.jpg',
 '26_1_1_20170112212042603.jpg.chip.jpg',
 '21222403_1983-09-29_2012.jpg',
 '593572_1960-09-04_2007.jpg',
 '32_1_1_20170116164948959.jpg.chip.jpg',
 '3776811_1938-01-05_1990.jpg',
 '42737444_1674-04-22_2014.jpg',
 '18_0_0_20170117140126568.jpg.chip.jpg',
 '20681612_1925-11-04_1947.jpg',
 '30_0_2_20170117141818577.jpg.chip.jpg',
 '004472.jpg',
 '22_1_1_20170112233701320.jpg.chip.jpg',
 '4334396_1972-09-07_1993.jpg',
 '27599355_1985-04-07_2007.jpg',
 '60_0_0_20170117160634375.jpg.chip.jpg',
 '6042299_1975-01-05_1970.jpg',
 '3214260_1968-09-12_2012.jpg',
 '15208001_1981-02-08_2012.jpg',
 '27_0_0_20170117175924156.jpg.chip.jpg',
 '23542494_1976-03-21_2012.jpg',
 '37867537_1965-02-11_1984.jpg',
 '37484443_1991-03-17_2013.jpg',
 '1_1_3_20161219224457585.jpg.chip.jpg',
 '18284846_1989-09-02_2010.jpg',
 '2456295_1969-10-14_2008.jpg',
 '20_1_0_20170103201733831.jpg.chip.jpg',
 '41334185_1991-06-24_2014.jpg',
 '17814123_1930-08-16_1974.jpg',
 '36873309_1959-12-01_2010.jpg',
 '65_1_0_20170110141141881.jpg.chip.jpg',
 '8484508_1981-12-31_2010.jpg',
 '26_0_2_20170119184102087.jpg.chip.jpg',
 '54_0_0_20170105163714131.jpg.chip.jpg',
 '31_0_1_20170113175408636.jpg.chip.jpg',
 '18676025_1969-06-11_2008.jpg',
 '26_1_4_20170117154137164.jpg.chip.jpg',
 '001075.jpg',
 '28_0_0_20170119194754322.jpg.chip.jpg',
 '73_0_0_20170120230233019.jpg.chip.jpg',
 '42412485_1931-09-03_1966.jpg',
 '26_1_4_20170103230455729.jpg.chip.jpg',
 '14013061_1977-11-07_2006.jpg',
 '72_0_2_20170112224532491.jpg.chip.jpg',
 '36_0_3_20170119194656666.jpg.chip.jpg',
 '1_0_2_20161219201442357.jpg.chip.jpg',
 '7285573_1929-05-07_1962.jpg',
 '001164.jpg',
 '6015495_1958-09-27_1950.jpg',
 '14996024_1982-09-16_2013.jpg',
 '2195339_1953-11-23_2008.jpg',
 '38480304_1911-04-12_2011.jpg',
 '42219298_1990-02-14_2014.jpg',
 '60_0_3_20170119205525232.jpg.chip.jpg',
 '2161453_1950-02-03_1973.jpg',
 '37464471_1957-05-22_1980.jpg',
 '38397939_1937-06-03_1960.jpg',
 '33319744_1992-04-26_1990.jpg',
 '3428686_1965-06-18_2015.jpg',
 '38_0_1_20170116003732637.jpg.chip.jpg',
 '2178459_1985-01-14_2008.jpg',
 '13_1_0_20170109204325267.jpg.chip.jpg',
 '20545506_1976-07-08_2009.jpg',
 '70_0_0_20170117173727535.jpg.chip.jpg',
 '005296.jpg',
 '26_1_2_20170116175903368.jpg.chip.jpg',
 '26_1_4_20170117201351845.jpg.chip.jpg',
 '24232963_1987-10-02_2012.jpg',
 '4_1_1_20170117202024231.jpg.chip.jpg',
 '5388344_1978-06-22_2012.jpg',
 '1497643_1916-08-14_1954.jpg',
 '004802.jpg',
 '37791562_1991-02-15_2014.jpg',
 '24_1_3_20170104223019607.jpg.chip.jpg',
 '11787967_1967-04-02_1960.jpg',
 '10_0_0_20170110225444491.jpg.chip.jpg',
 '6731123_1953-07-05_2011.jpg',
 '26897013_1991-06-25_2010.jpg',
 '002183.jpg',
 '42345_1900-01-27_1955.jpg',
 '863211_1930-06-01_1971.jpg',
 '22698827_1985-12-16_2013.jpg',
 '10838173_1990-02-01_2012.jpg',
 '8224016_1973-03-15_1970.jpg',
 '3660144_1972-04-13_1970.jpg',
 '42842300_1994-10-18_2014.jpg',
 '1515927_1980-09-26_2005.jpg',
 '307526_1964-09-08_2013.jpg',
 '6407782_1977-12-20_2007.jpg',
 '35937539_1992-08-08_2012.jpg',
 '33124994_1992-01-07_2011.jpg',
 '53_1_3_20170109141903857.jpg.chip.jpg',
 '40_0_2_20170116181214637.jpg.chip.jpg',
 '23_1_0_20170117144945297.jpg.chip.jpg',
 '003552.jpg',
 '23_1_3_20170117144858514.jpg.chip.jpg',
 '44059509_1928-01-08_1954.jpg',
 '24310807_1945-01-18_2009.jpg',
 '38634484_1993-07-30_2012.jpg',
 '8980506_1979-07-14_2009.jpg',
 '1042604_1956-06-04_2010.jpg',
 '64_0_0_20170104185632214.jpg.chip.jpg',
 '32_1_2_20170116191623170.jpg.chip.jpg',
 ...]

df_wiki_utk_appa.hist()

array([[<matplotlib.axes._subplots.AxesSubplot object at 0x7fcf163f9c88>]], dtype=object)

df_wiki_utk_appa['age'].value_counts()

26     2632
24     1593
28     1575
25     1573
30     1382
       ... 
98        2
97        2
101       1
111       1
103       1
Name: age, Length: 107, dtype: int64

df_wiki_utk_appa = df_wiki_utk_appa[df_wiki_utk_appa['age'] <= 100]
df_wiki_utk_appa = df_wiki_utk_appa[df_wiki_utk_appa['age'] > 0]

df_wiki_utk_appa['age'].min(), df_wiki_utk_appa['age'].max()

(1, 100)

df_wiki_utk_appa['age'] = df_wiki_utk_appa['age'].astype(int)

path_wiki_utk_appa = Path('../working/wiki_utk/')

np.random.seed(42)

data_wiki_small_src = (ImageList.from_df(df_wiki_utk_appa, path_wiki_utk_appa, cols=['full_path'], folder='.')
                   .split_by_rand_pct(0.2, seed=42)
                   .label_from_df(label_cls=FloatList))
                   
# data_wiki_big = ImageList.from_df(df_wiki_utk_appa, path_wiki_utk_appa, cols=['full_path'], folder='.').split_by_rand_pct(0.2, seed=42).label_from_df(label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=256).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)

data_wiki_small = (data_wiki_small_src.transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=128)
                   .databunch(bs=64,num_workers=0).normalize(imagenet_stats))

data_wiki_small.show_batch(4, figsize=(12,12))

Fastai Age Model

class AgeModel(nn.Module):
    def __init__(self):
        super().__init__()
        layers = list(models.resnet34(pretrained=True).children())[:-2]
        layers += [AdaptiveConcatPool2d(), Flatten()]
        #layers += [nn.BatchNorm1d(4096, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)]
        #layers += [nn.Dropout(p=0.60)]
        #layers += [nn.Linear(4096, 1024, bias=True), nn.ReLU(inplace=True)]
        #layers += [nn.BatchNorm1d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)]
        #layers += [nn.Dropout(p=0.60)]
        #layers += [nn.Linear(2048, 1024, bias=True), nn.ReLU(inplace=True)]
        #layers += [nn.BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)]
        #layers += [nn.Dropout(p=0.75)]
        #layers += [nn.Linear(1024, 256, bias=True), nn.ReLU(inplace=True)]
        #layers += [nn.BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)]
        #layers += [nn.Dropout(p=0.50)]
        #layers += [nn.Linear(512,256 , bias=True), nn.ReLU(inplace=True)]
        layers += [nn.BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)]
        layers += [nn.Dropout(p=0.50)]
        layers += [nn.Linear(1024, 512, bias=True), nn.ReLU(inplace=True)]
        layers += [nn.BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)]
        layers += [nn.Dropout(p=0.50)]
        layers += [nn.Linear(512, 16, bias=True), nn.ReLU(inplace=True)]
        layers += [nn.Linear(16,1)]
        self.agemodel = nn.Sequential(*layers)
    def forward(self, x):
        return self.agemodel(x).squeeze(-1)
          # could add 116*torch.sigmoid

model = AgeModel()

Downloading: "https://download.pytorch.org/models/resnet34-333f7ec4.pth" to /tmp/.cache/torch/checkpoints/resnet34-333f7ec4.pth
100%|██████████| 83.3M/83.3M [00:00<00:00, 92.9MB/s]

learn = Learner(data_wiki_small, model, model_dir = "/temp/model/", opt_func=opt_func, bn_wd=False, metrics=root_mean_squared_error,
               callback_fns=[ShowGraph]).mixup(stack_y=False, alpha=0.2)

learn.loss_func = L1LossFlat()

learn.split([model.agemodel[4],model.agemodel[6],model.agemodel[8]])

Learner(data=ImageDataBunch;

Train: LabelList (34909 items)
x: ImageList
Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128)
y: FloatList
52.0,34.0,36.0,23.0,78.0
Path: ../working/wiki_utk;

Valid: LabelList (8727 items)
x: ImageList
Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128)
y: FloatList
29.0,58.0,22.0,12.0,32.0
Path: ../working/wiki_utk;

Test: None, model=AgeModel(
  (agemodel): Sequential(
    (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace=True)
    (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
    (4): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (1): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (5): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (6): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (4): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (5): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (7): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (8): AdaptiveConcatPool2d(
      (ap): AdaptiveAvgPool2d(output_size=1)
      (mp): AdaptiveMaxPool2d(output_size=1)
    )
    (9): Flatten()
    (10): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (11): Dropout(p=0.5, inplace=False)
    (12): Linear(in_features=1024, out_features=512, bias=True)
    (13): ReLU(inplace=True)
    (14): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (15): Dropout(p=0.5, inplace=False)
    (16): Linear(in_features=512, out_features=16, bias=True)
    (17): ReLU(inplace=True)
    (18): Linear(in_features=16, out_features=1, bias=True)
  )
), opt_func=functools.partial(<class 'torch.optim.adam.Adam'>, betas=(0.9, 0.99), eps=1e-05), loss_func=L1LossFlat(), metrics=[<function root_mean_squared_error at 0x7fcf62a9bf28>], true_wd=True, bn_wd=False, wd=0.01, train_bn=True, path=PosixPath('../working/wiki_utk'), model_dir='/temp/model/', callback_fns=[functools.partial(<class 'fastai.basic_train.Recorder'>, add_time=True, silent=False), <class 'fastai.train.ShowGraph'>, functools.partial(<class 'fastai.callbacks.mixup.MixUpCallback'>, alpha=0.2, stack_x=False, stack_y=False)], callbacks=[], layer_groups=[Sequential(
  (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
), Sequential(
  (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (6): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): ReLU(inplace=True)
  (8): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (9): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (10): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): ReLU(inplace=True)
  (13): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (14): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (15): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (16): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): ReLU(inplace=True)
  (18): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (19): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (20): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (21): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (33): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (6): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (8): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (9): ReLU(inplace=True)
  (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (13): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (14): ReLU(inplace=True)
  (15): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (16): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (18): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (19): ReLU(inplace=True)
  (20): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (21): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (33): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (37): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (38): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (39): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (40): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (41): ReLU(inplace=True)
  (42): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (43): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (44): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (45): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (46): ReLU(inplace=True)
  (47): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (48): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): AdaptiveAvgPool2d(output_size=1)
  (1): AdaptiveMaxPool2d(output_size=1)
  (2): Flatten()
  (3): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (4): Dropout(p=0.5, inplace=False)
  (5): Linear(in_features=1024, out_features=512, bias=True)
  (6): ReLU(inplace=True)
  (7): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (8): Dropout(p=0.5, inplace=False)
  (9): Linear(in_features=512, out_features=16, bias=True)
  (10): ReLU(inplace=True)
  (11): Linear(in_features=16, out_features=1, bias=True)
)], add_time=True, silent=False, cb_fns_registered=False)

#learn = cnn_learner(data, resnet50, pretrained = True, model_dir = "/temp/model/",callback_fns=[ShowGraph])
#learn.loss_func = L1LossFlat()

import fastai
learn.freeze_to(-1)
learn.lr_find()
learn.recorder.plot(suggestion = True)

LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.
Min numerical gradient: 3.98E-02
Min loss divided by 10: 9.12E-02

lr = 2e-2

learn.fit_one_cycle(5, max_lr=slice(lr), wd=(1e-6, 1e-4, 1e-2, 1e-1), pct_start=0.5, callbacks=[SaveModelCallback(learn)])

Better model found at epoch 0 with valid_loss value: 9.252866744995117.
Better model found at epoch 1 with valid_loss value: 7.800601005554199.
Better model found at epoch 3 with valid_loss value: 6.707545757293701.
Better model found at epoch 4 with valid_loss value: 6.620563507080078.

learn.save('first_head_resnet34')
learn.load('first_head_resnet34')

Learner(data=ImageDataBunch;

Train: LabelList (34909 items)
x: ImageList
Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128)
y: FloatList
52.0,34.0,36.0,23.0,78.0
Path: ../working/wiki_utk;

Valid: LabelList (8727 items)
x: ImageList
Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128)
y: FloatList
29.0,58.0,22.0,12.0,32.0
Path: ../working/wiki_utk;

Test: None, model=AgeModel(
  (agemodel): Sequential(
    (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace=True)
    (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
    (4): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (1): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (5): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (6): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (4): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (5): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (7): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (8): AdaptiveConcatPool2d(
      (ap): AdaptiveAvgPool2d(output_size=1)
      (mp): AdaptiveMaxPool2d(output_size=1)
    )
    (9): Flatten()
    (10): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (11): Dropout(p=0.5, inplace=False)
    (12): Linear(in_features=1024, out_features=512, bias=True)
    (13): ReLU(inplace=True)
    (14): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (15): Dropout(p=0.5, inplace=False)
    (16): Linear(in_features=512, out_features=16, bias=True)
    (17): ReLU(inplace=True)
    (18): Linear(in_features=16, out_features=1, bias=True)
  )
), opt_func=functools.partial(<class 'torch.optim.adam.Adam'>, betas=(0.9, 0.99), eps=1e-05), loss_func=L1LossFlat(), metrics=[<function root_mean_squared_error at 0x7fcf62a9bf28>], true_wd=True, bn_wd=False, wd=0.01, train_bn=True, path=PosixPath('../working/wiki_utk'), model_dir='/temp/model/', callback_fns=[functools.partial(<class 'fastai.basic_train.Recorder'>, add_time=True, silent=False), <class 'fastai.train.ShowGraph'>, functools.partial(<class 'fastai.callbacks.mixup.MixUpCallback'>, alpha=0.2, stack_x=False, stack_y=False)], callbacks=[], layer_groups=[Sequential(
  (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
), Sequential(
  (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (6): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): ReLU(inplace=True)
  (8): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (9): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (10): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): ReLU(inplace=True)
  (13): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (14): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (15): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (16): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): ReLU(inplace=True)
  (18): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (19): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (20): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (21): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (33): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (6): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (8): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (9): ReLU(inplace=True)
  (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (13): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (14): ReLU(inplace=True)
  (15): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (16): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (18): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (19): ReLU(inplace=True)
  (20): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (21): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (33): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (37): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (38): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (39): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (40): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (41): ReLU(inplace=True)
  (42): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (43): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (44): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (45): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (46): ReLU(inplace=True)
  (47): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (48): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): AdaptiveAvgPool2d(output_size=1)
  (1): AdaptiveMaxPool2d(output_size=1)
  (2): Flatten()
  (3): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (4): Dropout(p=0.5, inplace=False)
  (5): Linear(in_features=1024, out_features=512, bias=True)
  (6): ReLU(inplace=True)
  (7): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (8): Dropout(p=0.5, inplace=False)
  (9): Linear(in_features=512, out_features=16, bias=True)
  (10): ReLU(inplace=True)
  (11): Linear(in_features=16, out_features=1, bias=True)
)], add_time=True, silent=False, cb_fns_registered=True)

learn.unfreeze()
learn.lr_find()
learn.recorder.plot(suggestion = True)

LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.
Min numerical gradient: 1.91E-06
Min loss divided by 10: 6.92E-07

learn.fit_one_cycle(5, max_lr=slice(1e-6, lr/5), wd=(1e-6, 1e-4, 1e-2, 1e-1), 
                    callbacks=[SaveModelCallback(learn)], pct_start=0.5)

Better model found at epoch 0 with valid_loss value: 7.008016109466553.
Better model found at epoch 1 with valid_loss value: 6.842098236083984.
Better model found at epoch 2 with valid_loss value: 6.805061340332031.
Better model found at epoch 3 with valid_loss value: 5.7170186042785645.
Better model found at epoch 4 with valid_loss value: 5.505085468292236.

x,y = next(iter(learn.data.train_dl))
x.shape

torch.Size([64, 3, 128, 128])

learn.save('first_body_resnet34')
learn.load('first_body_resnet34')

Learner(data=ImageDataBunch;

Train: LabelList (34909 items)
x: ImageList
Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128)
y: FloatList
52.0,34.0,36.0,23.0,78.0
Path: ../working/wiki_utk;

Valid: LabelList (8727 items)
x: ImageList
Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128)
y: FloatList
29.0,58.0,22.0,12.0,32.0
Path: ../working/wiki_utk;

Test: None, model=AgeModel(
  (agemodel): Sequential(
    (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace=True)
    (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
    (4): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (1): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (5): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (6): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (4): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (5): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (7): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (8): AdaptiveConcatPool2d(
      (ap): AdaptiveAvgPool2d(output_size=1)
      (mp): AdaptiveMaxPool2d(output_size=1)
    )
    (9): Flatten()
    (10): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (11): Dropout(p=0.5, inplace=False)
    (12): Linear(in_features=1024, out_features=512, bias=True)
    (13): ReLU(inplace=True)
    (14): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (15): Dropout(p=0.5, inplace=False)
    (16): Linear(in_features=512, out_features=16, bias=True)
    (17): ReLU(inplace=True)
    (18): Linear(in_features=16, out_features=1, bias=True)
  )
), opt_func=functools.partial(<class 'torch.optim.adam.Adam'>, betas=(0.9, 0.99), eps=1e-05), loss_func=L1LossFlat(), metrics=[<function root_mean_squared_error at 0x7fcf62a9bf28>], true_wd=True, bn_wd=False, wd=0.01, train_bn=True, path=PosixPath('../working/wiki_utk'), model_dir='/temp/model/', callback_fns=[functools.partial(<class 'fastai.basic_train.Recorder'>, add_time=True, silent=False), <class 'fastai.train.ShowGraph'>, functools.partial(<class 'fastai.callbacks.mixup.MixUpCallback'>, alpha=0.2, stack_x=False, stack_y=False)], callbacks=[], layer_groups=[Sequential(
  (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
), Sequential(
  (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (6): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): ReLU(inplace=True)
  (8): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (9): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (10): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): ReLU(inplace=True)
  (13): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (14): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (15): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (16): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): ReLU(inplace=True)
  (18): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (19): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (20): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (21): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (33): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (6): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (8): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (9): ReLU(inplace=True)
  (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (13): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (14): ReLU(inplace=True)
  (15): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (16): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (18): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (19): ReLU(inplace=True)
  (20): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (21): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (33): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (37): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (38): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (39): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (40): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (41): ReLU(inplace=True)
  (42): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (43): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (44): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (45): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (46): ReLU(inplace=True)
  (47): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (48): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): AdaptiveAvgPool2d(output_size=1)
  (1): AdaptiveMaxPool2d(output_size=1)
  (2): Flatten()
  (3): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (4): Dropout(p=0.5, inplace=False)
  (5): Linear(in_features=1024, out_features=512, bias=True)
  (6): ReLU(inplace=True)
  (7): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (8): Dropout(p=0.5, inplace=False)
  (9): Linear(in_features=512, out_features=16, bias=True)
  (10): ReLU(inplace=True)
  (11): Linear(in_features=16, out_features=1, bias=True)
)], add_time=True, silent=False, cb_fns_registered=True)

learn.show_results()

img = open_image('../input/picture4/PM_Modi_2015.jpg')
img

x = learn.predict(img)
x

(FloatItem 61.401455, tensor(61.4015), tensor(61.4015))

Image Resizing

data_wiki_big = (data_wiki_small_src.transform(tfms, size=256)
        .databunch(num_workers=0).normalize(imagenet_stats))

learn.data = data_wiki_big
data_wiki_big.train_ds[0][0].shape

torch.Size([3, 256, 256])

learn.freeze_to(-1)
learn.lr_find()
learn.recorder.plot(suggestion = True)

LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.
Min numerical gradient: 9.12E-07
Min loss divided by 10: 5.75E-03

lr = 5e-4

learn.fit_one_cycle(5, max_lr=slice(lr), wd=(1e-6, 1e-4, 1e-2, 1e-1), pct_start=0.5, callbacks=[SaveModelCallback(learn)])

Better model found at epoch 0 with valid_loss value: 6.402583122253418.
Better model found at epoch 1 with valid_loss value: 6.078464984893799.
Better model found at epoch 2 with valid_loss value: 5.939246654510498.
Better model found at epoch 3 with valid_loss value: 5.813475608825684.

learn.recorder.plot_lr()

learn.recorder.plot_losses()

learn.save('second_head_resnet34')
learn.load('second_head_resnet34')

Learner(data=ImageDataBunch;

Train: LabelList (34909 items)
x: ImageList
Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256)
y: FloatList
52.0,34.0,36.0,23.0,78.0
Path: ../working/wiki_utk;

Valid: LabelList (8727 items)
x: ImageList
Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256)
y: FloatList
29.0,58.0,22.0,12.0,32.0
Path: ../working/wiki_utk;

Test: None, model=AgeModel(
  (agemodel): Sequential(
    (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace=True)
    (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
    (4): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (1): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (5): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (6): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (4): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (5): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (7): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (8): AdaptiveConcatPool2d(
      (ap): AdaptiveAvgPool2d(output_size=1)
      (mp): AdaptiveMaxPool2d(output_size=1)
    )
    (9): Flatten()
    (10): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (11): Dropout(p=0.5, inplace=False)
    (12): Linear(in_features=1024, out_features=512, bias=True)
    (13): ReLU(inplace=True)
    (14): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (15): Dropout(p=0.5, inplace=False)
    (16): Linear(in_features=512, out_features=16, bias=True)
    (17): ReLU(inplace=True)
    (18): Linear(in_features=16, out_features=1, bias=True)
  )
), opt_func=functools.partial(<class 'torch.optim.adam.Adam'>, betas=(0.9, 0.99), eps=1e-05), loss_func=L1LossFlat(), metrics=[<function root_mean_squared_error at 0x7fcf62a9bf28>], true_wd=True, bn_wd=False, wd=0.01, train_bn=True, path=PosixPath('../working/wiki_utk'), model_dir='/temp/model/', callback_fns=[functools.partial(<class 'fastai.basic_train.Recorder'>, add_time=True, silent=False), <class 'fastai.train.ShowGraph'>, functools.partial(<class 'fastai.callbacks.mixup.MixUpCallback'>, alpha=0.2, stack_x=False, stack_y=False)], callbacks=[], layer_groups=[Sequential(
  (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
), Sequential(
  (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (6): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): ReLU(inplace=True)
  (8): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (9): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (10): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): ReLU(inplace=True)
  (13): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (14): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (15): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (16): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): ReLU(inplace=True)
  (18): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (19): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (20): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (21): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (33): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (6): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (8): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (9): ReLU(inplace=True)
  (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (13): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (14): ReLU(inplace=True)
  (15): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (16): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (18): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (19): ReLU(inplace=True)
  (20): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (21): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (33): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (37): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (38): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (39): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (40): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (41): ReLU(inplace=True)
  (42): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (43): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (44): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (45): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (46): ReLU(inplace=True)
  (47): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (48): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): AdaptiveAvgPool2d(output_size=1)
  (1): AdaptiveMaxPool2d(output_size=1)
  (2): Flatten()
  (3): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (4): Dropout(p=0.5, inplace=False)
  (5): Linear(in_features=1024, out_features=512, bias=True)
  (6): ReLU(inplace=True)
  (7): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (8): Dropout(p=0.5, inplace=False)
  (9): Linear(in_features=512, out_features=16, bias=True)
  (10): ReLU(inplace=True)
  (11): Linear(in_features=16, out_features=1, bias=True)
)], add_time=True, silent=False, cb_fns_registered=True)

img = open_image('../input/picture4/PM_Modi_2015.jpg')
img

learn.predict(img)

(FloatItem 61.38315, tensor(61.3831), tensor(61.3831))

x,y = next(iter(learn.data.train_dl))
x.shape

torch.Size([64, 3, 256, 256])

learn.unfreeze()
learn.lr_find()
learn.recorder.plot(suggestion = True)

LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.
Min numerical gradient: 9.12E-07
Min loss divided by 10: 3.02E-06

learn.fit_one_cycle(5, max_lr=slice(1e-6,1e-4), wd=(1e-6, 1e-4, 1e-2, 1e-1), callbacks=[SaveModelCallback(learn)], pct_start=0.5)

Better model found at epoch 0 with valid_loss value: 5.71042013168335.
Better model found at epoch 1 with valid_loss value: 5.421929359436035.
Better model found at epoch 2 with valid_loss value: 5.245903968811035.
Better model found at epoch 3 with valid_loss value: 5.199910640716553.
Better model found at epoch 4 with valid_loss value: 5.103240489959717.

learn.save('second_body_resnet34')
learn.load('second_body_resnet34')

Learner(data=ImageDataBunch;

Train: LabelList (34909 items)
x: ImageList
Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256)
y: FloatList
52.0,34.0,36.0,23.0,78.0
Path: ../working/wiki_utk;

Valid: LabelList (8727 items)
x: ImageList
Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256)
y: FloatList
29.0,58.0,22.0,12.0,32.0
Path: ../working/wiki_utk;

Test: None, model=AgeModel(
  (agemodel): Sequential(
    (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace=True)
    (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
    (4): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (1): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (5): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (6): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (4): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (5): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (7): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (8): AdaptiveConcatPool2d(
      (ap): AdaptiveAvgPool2d(output_size=1)
      (mp): AdaptiveMaxPool2d(output_size=1)
    )
    (9): Flatten()
    (10): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (11): Dropout(p=0.5, inplace=False)
    (12): Linear(in_features=1024, out_features=512, bias=True)
    (13): ReLU(inplace=True)
    (14): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (15): Dropout(p=0.5, inplace=False)
    (16): Linear(in_features=512, out_features=16, bias=True)
    (17): ReLU(inplace=True)
    (18): Linear(in_features=16, out_features=1, bias=True)
  )
), opt_func=functools.partial(<class 'torch.optim.adam.Adam'>, betas=(0.9, 0.99), eps=1e-05), loss_func=L1LossFlat(), metrics=[<function root_mean_squared_error at 0x7fcf62a9bf28>], true_wd=True, bn_wd=False, wd=0.01, train_bn=True, path=PosixPath('../working/wiki_utk'), model_dir='/temp/model/', callback_fns=[functools.partial(<class 'fastai.basic_train.Recorder'>, add_time=True, silent=False), <class 'fastai.train.ShowGraph'>, functools.partial(<class 'fastai.callbacks.mixup.MixUpCallback'>, alpha=0.2, stack_x=False, stack_y=False)], callbacks=[], layer_groups=[Sequential(
  (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
), Sequential(
  (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (6): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): ReLU(inplace=True)
  (8): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (9): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (10): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): ReLU(inplace=True)
  (13): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (14): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (15): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (16): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): ReLU(inplace=True)
  (18): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (19): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (20): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (21): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (33): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (6): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (8): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (9): ReLU(inplace=True)
  (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (13): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (14): ReLU(inplace=True)
  (15): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (16): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (18): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (19): ReLU(inplace=True)
  (20): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (21): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (33): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (37): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (38): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (39): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (40): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (41): ReLU(inplace=True)
  (42): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (43): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (44): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (45): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (46): ReLU(inplace=True)
  (47): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (48): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): AdaptiveAvgPool2d(output_size=1)
  (1): AdaptiveMaxPool2d(output_size=1)
  (2): Flatten()
  (3): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (4): Dropout(p=0.5, inplace=False)
  (5): Linear(in_features=1024, out_features=512, bias=True)
  (6): ReLU(inplace=True)
  (7): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (8): Dropout(p=0.5, inplace=False)
  (9): Linear(in_features=512, out_features=16, bias=True)
  (10): ReLU(inplace=True)
  (11): Linear(in_features=16, out_features=1, bias=True)
)], add_time=True, silent=False, cb_fns_registered=True)

learn.show_results()

img = open_image('../input/picture4/PM_Modi_2015.jpg')
img

x = learn.predict(img)
x

(FloatItem 63.15648, tensor(63.1565), tensor(63.1565))

learn.save('third_body_resnet34')
learn.load('third_body_resnet34')

Learner(data=ImageDataBunch;

Train: LabelList (34909 items)
x: ImageList
Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256)
y: FloatList
52.0,34.0,36.0,23.0,78.0
Path: ../working/wiki_utk;

Valid: LabelList (8727 items)
x: ImageList
Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256)
y: FloatList
29.0,58.0,22.0,12.0,32.0
Path: ../working/wiki_utk;

Test: None, model=AgeModel(
  (agemodel): Sequential(
    (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace=True)
    (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
    (4): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (1): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (5): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (6): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (4): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (5): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (7): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (8): AdaptiveConcatPool2d(
      (ap): AdaptiveAvgPool2d(output_size=1)
      (mp): AdaptiveMaxPool2d(output_size=1)
    )
    (9): Flatten()
    (10): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (11): Dropout(p=0.5, inplace=False)
    (12): Linear(in_features=1024, out_features=512, bias=True)
    (13): ReLU(inplace=True)
    (14): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (15): Dropout(p=0.5, inplace=False)
    (16): Linear(in_features=512, out_features=16, bias=True)
    (17): ReLU(inplace=True)
    (18): Linear(in_features=16, out_features=1, bias=True)
  )
), opt_func=functools.partial(<class 'torch.optim.adam.Adam'>, betas=(0.9, 0.99), eps=1e-05), loss_func=L1LossFlat(), metrics=[<function root_mean_squared_error at 0x7fcf62a9bf28>], true_wd=True, bn_wd=False, wd=0.01, train_bn=True, path=PosixPath('../working/wiki_utk'), model_dir='/temp/model/', callback_fns=[functools.partial(<class 'fastai.basic_train.Recorder'>, add_time=True, silent=False), <class 'fastai.train.ShowGraph'>, functools.partial(<class 'fastai.callbacks.mixup.MixUpCallback'>, alpha=0.2, stack_x=False, stack_y=False)], callbacks=[], layer_groups=[Sequential(
  (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
), Sequential(
  (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (6): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): ReLU(inplace=True)
  (8): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (9): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (10): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): ReLU(inplace=True)
  (13): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (14): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (15): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (16): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): ReLU(inplace=True)
  (18): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (19): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (20): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (21): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (33): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU(inplace=True)
  (3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (6): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (7): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (8): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (9): ReLU(inplace=True)
  (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (11): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (13): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (14): ReLU(inplace=True)
  (15): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (16): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (17): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (18): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (19): ReLU(inplace=True)
  (20): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (21): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (22): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (23): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (24): ReLU(inplace=True)
  (25): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (26): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (27): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (28): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (29): ReLU(inplace=True)
  (30): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (31): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (32): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
  (33): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (34): ReLU(inplace=True)
  (35): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (36): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (37): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
  (38): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (39): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (40): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (41): ReLU(inplace=True)
  (42): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (43): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (44): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (45): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (46): ReLU(inplace=True)
  (47): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
  (48): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
  (0): AdaptiveAvgPool2d(output_size=1)
  (1): AdaptiveMaxPool2d(output_size=1)
  (2): Flatten()
  (3): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (4): Dropout(p=0.5, inplace=False)
  (5): Linear(in_features=1024, out_features=512, bias=True)
  (6): ReLU(inplace=True)
  (7): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (8): Dropout(p=0.5, inplace=False)
  (9): Linear(in_features=512, out_features=16, bias=True)
  (10): ReLU(inplace=True)
  (11): Linear(in_features=16, out_features=1, bias=True)
)], add_time=True, silent=False, cb_fns_registered=True)

Fastai Hooks

class SaveFeatures():
    features=None
    def __init__(self, m): 
        self.hook = m.register_forward_hook(self.hook_fn)
        self.features = None
    def hook_fn(self, module, input, output): 
        out = output.detach().cpu().numpy()
        if isinstance(self.features, type(None)):
            self.features = out
        else:
            self.features = np.row_stack((self.features, out))
    def remove(self): 
        self.hook.remove()

learn.model

AgeModel(
  (agemodel): Sequential(
    (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace=True)
    (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
    (4): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (1): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (5): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (6): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (3): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (4): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (5): BasicBlock(
        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (7): Sequential(
      (0): BasicBlock(
        (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (downsample): Sequential(
          (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
          (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (2): BasicBlock(
        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
    )
    (8): AdaptiveConcatPool2d(
      (ap): AdaptiveAvgPool2d(output_size=1)
      (mp): AdaptiveMaxPool2d(output_size=1)
    )
    (9): Flatten()
    (10): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (11): Dropout(p=0.5, inplace=False)
    (12): Linear(in_features=1024, out_features=512, bias=True)
    (13): ReLU(inplace=True)
    (14): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (15): Dropout(p=0.5, inplace=False)
    (16): Linear(in_features=512, out_features=16, bias=True)
    (17): ReLU(inplace=True)
    (18): Linear(in_features=16, out_features=1, bias=True)
  )
)

# Second last layer of the model
learn.model.agemodel[16]

Linear(in_features=512, out_features=16, bias=True)

len(data_wiki_big.train_ds.items), len(data_wiki_big.valid_ds.items)

(34909, 8727)

sf = SaveFeatures(learn.model.agemodel[16])

_= learn.get_preds(data_wiki_big.train_ds)
_= learn.get_preds(DatasetType.Valid)

len(sf.features)

43636

img_path = [str(x) for x in (list(data_wiki_big.train_ds.items) +list(data_wiki_big.valid_ds.items))]
label = [x for x in (list(data_wiki_big.train_ds.y.items) +list(data_wiki_big.valid_ds.y.items))]
label_id = [x for x in (list(data_wiki_big.train_ds.y.items) +list(data_wiki_big.valid_ds.y.items))]

df_new = pd.DataFrame({'img_path': img_path, 'label': label, 'label_id': label_id})
df_new

array = np.array(sf.features)

x=array.tolist()

df_new['img_repr'] = x

df_new

df_new.shape

(43636, 4)

Annoy

from annoy import AnnoyIndex

f = len(df_new['img_repr'][0])
t = AnnoyIndex(f, metric='euclidean')

ntree = 1000

for i, vector in enumerate(df_new['img_repr']):
    t.add_item(i, vector)
_  = t.build(ntree)

import time
def get_similar_images_annoy(img_index, num):
    start = time.time()
    base_img_id, base_vector, base_label  = df_new.iloc[img_index, [0,3,1]]
    similar_img_ids = t.get_nns_by_item(img_index, num)
    end = time.time()
    print(f'{(end - start) * 1000} ms')
    return base_img_id, base_label, df_new.iloc[similar_img_ids]

num = 89
base_image, base_label, similar_images_df = get_similar_images_annoy(890, num)

70.15395164489746 ms

print(base_label)
open_image(base_image)

21.0

similar_images_df

data_src = (ImageList.from_df(similar_images_df, path=".",cols=['img_path'], folder='.' )
        .split_none()
        .label_from_df(cols=['label'],label_cls=FloatList))

data = (data_src.transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=256)
                   .databunch(bs=num,num_workers=0).normalize(imagenet_stats))

data.show_batch(2)

	dob	photo_taken	full_path	gender	name	face_location	face_score	second_face_score	celeb_names	celeb_id
0	723671	2009	[17/10000217_1981-05-05_2009.jpg]	1.0	[Sami Jauhojärvi]	[[111.29109473290997, 111.29109473290997, 252....	4.300962	NaN	NaN	NaN
1	703186	1964	[48/10000548_1925-04-04_1964.jpg]	1.0	[Dettmar Cramer]	[[252.48330229530742, 126.68165114765371, 354....	2.645639	1.949248	NaN	NaN
2	711677	2008	[12/100012_1948-07-03_2008.jpg]	1.0	[Marc Okrand]	[[113.52, 169.83999999999997, 366.08, 422.4]]	4.329329	NaN	NaN	NaN
3	705061	1961	[65/10001965_1930-05-23_1961.jpg]	1.0	[Aleksandar Matanović]	[[1, 1, 634, 440]]	-inf	NaN	NaN	NaN
4	720044	2012	[16/10002116_1971-05-31_2012.jpg]	0.0	[Diana Damrau]	[[171.61031405173117, 75.57451239763239, 266.7...	3.408442	NaN	NaN	NaN

	dob	photo_taken	full_path	gender	name	face_location	face_score	second_face_score	celeb_names	celeb_id
0	693726	1968	[01/nm0000001_rm124825600_1899-5-10_1968.jpg]	1.0	[Fred Astaire]	[[1072.926, 161.838, 1214.7839999999999, 303.6...	1.459693	1.118973	['Lee' George Quinones]	6488
1	693726	1970	[01/nm0000001_rm3343756032_1899-5-10_1970.jpg]	1.0	[Fred Astaire]	[[477.184, 100.352, 622.592, 245.76]]	2.543198	1.852008	['Weird Al' Yankovic]	6488
2	693726	1968	[01/nm0000001_rm577153792_1899-5-10_1968.jpg]	1.0	[Fred Astaire]	[[114.96964308962852, 114.96964308962852, 451....	3.455579	2.985660	[2 Chainz]	6488
3	693726	1968	[01/nm0000001_rm946909184_1899-5-10_1968.jpg]	1.0	[Fred Astaire]	[[622.8855056426588, 424.21750383700805, 844.3...	1.872117	NaN	[50 Cent]	6488
4	693726	1968	[01/nm0000001_rm980463616_1899-5-10_1968.jpg]	1.0	[Fred Astaire]	[[1013.8590023603723, 233.8820422075853, 1201....	1.158766	NaN	[A Martinez]	6488

	x	y
0	39_0_1_20170116143117964.jpg.chip.jpg	39.0
1	54_0_3_20170104202315035.jpg.chip.jpg	54.0
2	1_0_2_20161219154715885.jpg.chip.jpg	1.0
3	30_1_2_20170116192313635.jpg.chip.jpg	30.0
4	41_1_0_20170117133203334.jpg.chip.jpg	41.0

epoch	train_loss	valid_loss	root_mean_squared_error	time
0	9.605057	9.252867	13.258863	07:27
1	8.392913	7.800601	11.307045	07:26
2	7.934024	8.536113	12.432997	07:25
3	7.214382	6.707546	9.728629	07:25
4	7.115435	6.620564	9.895959	07:26

epoch	train_loss	valid_loss	root_mean_squared_error	time
0	7.085314	7.008016	10.614280	07:32
1	7.218247	6.842098	9.879888	07:31
2	6.800269	6.805061	9.620695	07:30
3	6.229340	5.717019	8.594589	07:30
4	6.011308	5.505085	8.282001	07:31

	full_path	age
0	17/10000217_1981-05-05_2009.jpg	28
2	12/100012_1948-07-03_2008.jpg	60
5	02/10002702_1960-11-09_2012.jpg	52
6	41/10003541_1937-09-27_1971.jpg	34
10	99/10004299_1908-08-19_1950.jpg	42

	full_path	age
7	02/nm0000002_rm1363385088_1924-9-16_2004.jpg	80
12	02/nm0000002_rm221957120_1924-9-16_1974.jpg	50
14	02/nm0000002_rm238734336_1924-9-16_1974.jpg	50
15	02/nm0000002_rm2585828096_1924-9-16_2006.jpg	82
18	02/nm0000002_rm2769394176_1924-9-16_2006.jpg	82

	file_name	real_age
0	000000.jpg	4
1	000001.jpg	18
2	000002.jpg	80
3	000003.jpg	50
4	000004.jpg	17

	full_path	age
0	10000217_1981-05-05_2009.jpg	28
2	100012_1948-07-03_2008.jpg	60
5	10002702_1960-11-09_2012.jpg	52
6	10003541_1937-09-27_1971.jpg	34
10	10004299_1908-08-19_1950.jpg	42

epoch	train_loss	valid_loss	root_mean_squared_error	time
0	7.837380	6.402583	9.601757	14:34
1	7.332215	6.078465	9.117067	14:26
2	7.223328	5.939247	8.911067	14:18
3	6.803191	5.813476	8.721518	14:18
4	7.061768	5.878111	8.838454	14:21

epoch	train_loss	valid_loss	root_mean_squared_error	time
0	6.700283	5.710420	8.648839	14:52
1	6.323674	5.421929	8.128598	15:00
2	6.052965	5.245904	7.984351	14:55
3	5.895456	5.199911	7.924792	14:54
4	5.728145	5.103240	7.743491	14:55

	img_path	label	label_id
0	../working/wiki_utk/./10002702_1960-11-09_2012...	52.0	52.0
1	../working/wiki_utk/./10003541_1937-09-27_1971...	34.0	34.0
2	../working/wiki_utk/./1000456_1933-06-12_1969.jpg	36.0	36.0
3	../working/wiki_utk/./10004882_1987-05-16_2010...	23.0	23.0
4	../working/wiki_utk/./1000781_1930-07-07_2008.jpg	78.0	78.0
...	...	...	...
43631	../working/wiki_utk/./007148.jpg	20.0	20.0
43632	../working/wiki_utk/./4948808_1983-11-16_2007.jpg	24.0	24.0
43633	../working/wiki_utk/./6_1_4_20170119204208503....	6.0	6.0
43634	../working/wiki_utk/./26_0_4_20170117154446109...	26.0	26.0
43635	../working/wiki_utk/./18_0_4_20170103201305015...	18.0	18.0

	img_path	label	label_id	img_repr
890	../working/wiki_utk/./950234_1941-03-06_1962.jpg	21.0	21.0	[44.051395416259766, -8.610692024230957, -7.61...
10285	../working/wiki_utk/./1575279_1978-12-28_2014.jpg	36.0	36.0	[44.23878479003906, -8.540730476379395, -7.615...
40921	../working/wiki_utk/./23465846_1989-07-10_2010...	21.0	21.0	[44.385948181152344, -8.653063774108887, -7.68...
11172	../working/wiki_utk/./4182570_1978-06-26_2012.jpg	34.0	34.0	[44.501441955566406, -8.626776695251465, -7.64...
34427	../working/wiki_utk/./007004.jpg	27.0	27.0	[44.107200622558594, -8.570219993591309, -7.66...
...	...	...	...	...
3505	../working/wiki_utk/./25239834_1992-01-23_2015...	23.0	23.0	[44.192176818847656, -8.403630256652832, -7.45...
40902	../working/wiki_utk/./1691972_1933-02-18_1968.jpg	35.0	35.0	[45.43599319458008, -8.536643981933594, -7.595...
12401	../working/wiki_utk/./1190101_1979-11-13_2010.jpg	31.0	31.0	[45.04903793334961, -8.50577449798584, -7.5224...
40885	../working/wiki_utk/./4094157_1990-03-01_2009.jpg	19.0	19.0	[43.83957290649414, -8.407843589782715, -7.513...
13491	../working/wiki_utk/./589278_1897-01-23_1942.jpg	45.0	45.0	[45.358192443847656, -8.5722074508667, -7.6586...